cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Setup Git Integration via REST API

gbrueckl
Contributor II

We are currently setting up CI/CD for our Databricks workspace using Databricks Repos following the approach described in the offical docs: https://docs.databricks.com/repos.html#best-practices-for-integrating-databricks-repos-with-cicd-wor...

Obviously for our CI/CD pipelines we want to use service account(we are on Azure using AAD Service Principals) and not use regular user accounts.

Unfortunately, there is no way to configure the Git integration in a scripted way (e.g. via Repos REST API)

the docs further mention (https://docs.databricks.com/repos.html#configure-your-git-integration-with-databricks😞

For Azure DevOps, Git integration does not support Azure Active Directory tokens. You must use an Azure DevOps personal access token.

Which is another pity as ideally it would just pass the AAD token used to authenticate with the Databricks REST API further down to the Azure DevOps Repo

so how is this supposed to work as a fully automated CI/CD pipeline?

1 ACCEPTED SOLUTION

Accepted Solutions

Hi @Gerhard Brueckl​ ,

Any Databricks user can participate in the Ideas Portal. I can hover through the Azure portal also.

image 

imageYou may have to filter the page with the newest idea.

imageYou can hover over " Post A New Idea" . When you do so you will find this page wherein you can share your ideas.

imageTalking about the updates, this page has the latest idea 2 days ago from an Azure Databricks user.

Please feel free to reach out in case you have any more questions.

View solution in original post

16 REPLIES 16

Atanu
Esteemed Contributor

gbrueckl
Contributor II

Thanks Atanu but none of these covers the process of setting up the connection and authentication to the git repo so I guess this is just not supported?

will this may come in the future?

Atanu
Esteemed Contributor

@Gerhard Brueckl​  could you please raise a feature request from https://docs.databricks.com/resources/ideas.html. Thank you!

gbrueckl
Contributor II

I am on Azure and when I go to the ideas-page of azure (https://docs.microsoft.com/en-gb/azure/databricks/resources/ideas) I get a 404 Error ...

If I open the Feedback-page directly from my DAtabricks Workspace it brings me here https://feedback.azure.com/d365community/forum/2efba7dc-ef24-ec11-b6e6-000d3a4f0da0 where most of the ideas are a couple of years old and have 0 comments (some of them have already been fixed but never got updated)

Hi @Gerhard Brueckl​ , You can use Ideas portal for requesting a new feature.

again, as I am on Azure I cannot login to the ideas portal is it is solely for AWS customers

Hi @Gerhard Brueckl​ ,

Any Databricks user can participate in the Ideas Portal. I can hover through the Azure portal also.

image 

imageYou may have to filter the page with the newest idea.

imageYou can hover over " Post A New Idea" . When you do so you will find this page wherein you can share your ideas.

imageTalking about the updates, this page has the latest idea 2 days ago from an Azure Databricks user.

Please feel free to reach out in case you have any more questions.

Atanu
Esteemed Contributor

@Gerhard Brueckl​  ok so for azure workspace you can share your request as per below -

Screenshot 2022-03-09 at 9.28.44 PMThe feedback option is the right place.

Thank you.

Thanks, Atanu, please read my previous comments about the state of the page where this link directs you

does not look like that page got updated in the last 2 years

Hi @Gerhard Brueckl​ , This link takes you to the page where the data is filtered with Ideas with most votes and that's the reason you may see the ideas which are 3 years ago.

If you filter out the Ideas with newest, you'll see the latest Ideas.

image

Hi @Kaniz Fatma​ ,

sure, I just scrolled through all pages there and found a total of < 10 comments (non from Microsoft or Databricks) , a bunch of "ideas" that are already implemented but never got updated and a lot of ideas from Microsoft itself making it look like an orphaned backlog.

dont you share the impression that that page is dead?

A similar idea to my original question was also already submitted 2 months ago - also not comments, no nothing - still, I upvoted it but honestly I do not expect anything to happen

https://feedback.azure.com/d365community/idea/0db9b406-946b-ec11-a81b-6045bd7956bb

-gerhard

Hi @Gerhard Brueckl​ , I sincerely apologise for the experience. Let me get back to you by hovering on the same. Thank you for bringing this to us.

Hi @Gerhard Brueckl​ , We're working on it. Please bear with me for the delay. Apologies.

New1
New Contributor II

Hi, how can i trigger a job externally using Github actions?

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group