cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

thib
by New Contributor III
  • 5594 Views
  • 4 replies
  • 2 kudos

Can we use multiple git repos for a job running multiple tasks?

I have a job running multiple tasks :Task 1 runs a machine learning pipeline from git repo 1Task 2 runs an ETL pipeline from git repo 1Task 2 is actually a generic pipeline and should not be checked in repo 1, and will be made available in another re...

image
  • 5594 Views
  • 4 replies
  • 2 kudos
Latest Reply
tors_r_us
New Contributor
  • 2 kudos

Had this same problem. Fix was to have two workflows with no triggers, each pointing to the respective git repo. Then setup a 3rd workflow with appropriate triggers/schedule which calls the first 2 workflows. A workflow can run other workflows. 

  • 2 kudos
3 More Replies
chris_kimmel
by New Contributor II
  • 953 Views
  • 0 replies
  • 2 kudos

Bug report: Switching branches duplicates cells

I'm using Databricks' support for GitHub repos. When I switch from one branch to another while a notebook is open, it messes up my notebook. Specifically, every notebook cell appears twice after switching branches.

  • 953 Views
  • 0 replies
  • 2 kudos
alefred
by New Contributor II
  • 4654 Views
  • 3 replies
  • 2 kudos

Resolved! Automation creation of Databricks Top Level Repositories

I'm looking to automate the creation of Top Level repositories in Databricks however isn't possible using cli or API if this repo is private repository(Azure DevOps Repository) because require setup the token in user setting.databricks repos create \...

  • 4654 Views
  • 3 replies
  • 2 kudos
Latest Reply
Atanu
Databricks Employee
  • 2 kudos

We have API availabe for repos https://docs.databricks.com/dev-tools/api/latest/repos.html#operation/get-repos also we are currently supporting SP. Step 1: As an admin Create a Service PrincipalUse this API SCIM API 2.0 (ServicePrincipals) | Databric...

  • 2 kudos
2 More Replies
thushar
by Contributor
  • 6399 Views
  • 0 replies
  • 0 kudos

How to install wheel package from git repo

Using VS code for development and a wheel package is created for shipment.We put this wheel package in Azure data lake storage and ADB notebook accessed this wheel package and installed it in the cluster. It is working fine. But instead of keeping th...

image.png
  • 6399 Views
  • 0 replies
  • 0 kudos
Labels