topic DABs require library dependancies from GitHub private repository. in Data Engineering

DABs require library dependancies from GitHub private repository.

Aatma — Wed, 21 Aug 2024 00:50:35 GMT

developing a python wheel file using DABs which require library dependancies from GitHub private repository. Please help me understand how to setup the git user and token in the resource.yml file and how to authenticate the GitHub package.

pip install git+https://<your-github-token>@github.com/<your-username>/<your-repo>.git

Re: DABs require library dependancies from GitHub private repository.

Satyadeepak — Thu, 16 Jan 2025 18:05:37 GMT

To install dependencies from a private GitHub repository in a Databricks Asset Bundle, you need to set up the GitHub user and token in the resource.yml file and authenticate the GitHub package. Here are the steps:

Generate a GitHub Personal Access Token:
- Go to GitHub and navigate to Settings > Developer settings > Personal access tokens.
- Click Generate new token.
- Select the scopes you need (e.g., repo for full control of private repositories).
- Click Generate token and copy the token.
Set Up the GitHub User and Token in resource.yml:
- In your resource.yml file, add the GitHub user and token as environment variables. For example:
```
resources:
  jobs:
    my-job:
      name:
```

Re: DABs require library dependancies from GitHub private repository.

sandy311 — Sun, 06 Apr 2025 19:17:20 GMT

Could you please give a detailed example?

how to define env varaibles? BUNDLE_VAR?

Re: DABs require library dependancies from GitHub private repository.

JGoedhart — Wed, 16 Apr 2025 11:27:26 GMT

@Satyadeepak : I'm also curious how to configure this. What are the github user and token environment variables names databricks uses to configure github SSH authentication? Can't find them in the documentation