Bitbucket Cloud Repo Integration with Token

[Carsten03]

Hey,

I am using Bitbucket Cloud and I want to connect my repository to Databricks. I am able to connect with my personal app password but what I am looking for is an authentication of a technical user.

I need the integration to point to my dbt repo, which will run as a workflow task and I don't want to use my personal git credentials for that because the service principal is the owner of the pipeline.

Is there a way of using non-personal access tokens to authenticate or is the only way creating a new technical user on Bitbucket?

Thank you!

[Kaniz · Accepted solution]

Hi @Carsten03, Let’s explore the options for authenticating your Databricks integration with Bitbucket Cloud.

1. Databricks GitHub App (Recommended):

   • If you’re using a hosted GitHub account, consider using the Databricks GitHub App for user authentication instead of Personal Access Tokens (PATs).
   • Benefits of using the Databricks GitHub App:
     • It uses OAuth 2.0 for user authentication, ensuring strong security.
     • OAuth 2.0 repo traffic is encrypted.
     • Integration is easier and doesn’t require individual tracking of tokens.
     • Token renewal is handled automatically.
     • You can scope the integration to specific attached Git repos, allowing granular control over access.
   • To set this up:
     • Link your GitHub account in Databricks by going to User Settings > Linked accounts.
     • Authorize the Databricks GitHub app to act on your behalf for Git operations in Repos (such as cloning a repository).
     • Install and configure the Databricks GitHub app on the GitHub repositories you want to access from Databricks Repos.
     • Note that this feature is not supported in GitHub Enterprise Server; use a personal access token ins...¹.

2. Bitbucket Cloud Repository Access Token:

   • To avoid tying git credentials to an individual user, you can use a Repository Access Token from Bitbucket Cloud.
   • However, the Databricks API does not explicitly define how a Repository Access Token should fit, as there is no defined git_username other than the <hash>@bots.
   • If you choose this approach, create an app password in Bitbucket Cloud that allows access to your repositories. Record the password.
   • In Databricks, go to User Settings > Linked accounts and enter this password to link your Bitbucket ...².

3. Databricks Personal Access Token (PAT):

   • If you prefer to use a PAT, you can set up authentication for the Databricks CLI using a Databricks PAT.
   • First, create a PAT for your Databricks workspace user. Then configure the Databricks CLI to use this token³.

[Carsten03]

Hi @Kaniz,

thank you for your response! With this link you provided, I was able to authenticate with Bitbucket Cloud. The solution was to use x-token-auth as a username. I have tried with the generated email address before which didn't work. Thank you! Now I am facing the next issue.

I wanted to use the repository access to run dbt via workflows. I have added the git credential via terraform and checked with the cli that it is available, but I now I get the following error message:

[RepositoryCheckoutFailed] Failed to checkout Git repository: PERMISSION_DENIED: Missing Git provider credentials. Go to User Settings > Git Integration to set up your Git credentials

I want to run the workflow with a service principal as an owner and therefore don't want to set the git credentials for my personal user. How can the service principal get access to the git credential / repository?

[Carsten03]

I have created a new topic for this here: https://community.databricks.com/t5/administration-architecture/run-workflow-using-git-integration-w...