cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

thib
by New Contributor III
  • 4945 Views
  • 3 replies
  • 2 kudos

Can we use multiple git repos for a job running multiple tasks?

I have a job running multiple tasks :Task 1 runs a machine learning pipeline from git repo 1Task 2 runs an ETL pipeline from git repo 1Task 2 is actually a generic pipeline and should not be checked in repo 1, and will be made available in another re...

image
  • 4945 Views
  • 3 replies
  • 2 kudos
Latest Reply
trijit
New Contributor II
  • 2 kudos

The way to go about this would be to create Databricks repos in the workspace and then use that in the task formation. This way we can refer multiple repos in different tasks.

  • 2 kudos
2 More Replies
chris_kimmel
by New Contributor II
  • 865 Views
  • 0 replies
  • 2 kudos

Bug report: Switching branches duplicates cells

I'm using Databricks' support for GitHub repos. When I switch from one branch to another while a notebook is open, it messes up my notebook. Specifically, every notebook cell appears twice after switching branches.

  • 865 Views
  • 0 replies
  • 2 kudos
alefred
by New Contributor II
  • 4364 Views
  • 3 replies
  • 2 kudos

Resolved! Automation creation of Databricks Top Level Repositories

I'm looking to automate the creation of Top Level repositories in Databricks however isn't possible using cli or API if this repo is private repository(Azure DevOps Repository) because require setup the token in user setting.databricks repos create \...

  • 4364 Views
  • 3 replies
  • 2 kudos
Latest Reply
Atanu
Databricks Employee
  • 2 kudos

We have API availabe for repos https://docs.databricks.com/dev-tools/api/latest/repos.html#operation/get-repos also we are currently supporting SP. Step 1: As an admin Create a Service PrincipalUse this API SCIM API 2.0 (ServicePrincipals) | Databric...

  • 2 kudos
2 More Replies
thushar
by Contributor
  • 5784 Views
  • 0 replies
  • 0 kudos

How to install wheel package from git repo

Using VS code for development and a wheel package is created for shipment.We put this wheel package in Azure data lake storage and ADB notebook accessed this wheel package and installed it in the cluster. It is working fine. But instead of keeping th...

image.png
  • 5784 Views
  • 0 replies
  • 0 kudos
Labels