Private package installation using DAB on job cluster

[sandy311]

I'm using a .whl job to upload and install a package on a job cluster using DAB. However, I'm facing an issue with a private package from Azure Artifacts. It works fine when running in CI or Azure DevOps pipelines because the PAT token is available as an environment variable. But when running the job through DAB, it fails to install the dependent private package due to authentication issues and im using `uv` to build package and dependency

[Brahmareddy]

Hi sandy311,

How are you doing today?, As per my understanding, It sounds like you're really close—this issue is happening because when your job runs through Databricks Asset Bundles (DAB) on a job cluster, it doesn’t automatically have access to the PAT token you use in your CI pipeline to authenticate with Azure Artifacts. Even though it works in CI, the environment in DAB is isolated, so you’ll need to pass the token securely to your job cluster at runtime. The simplest fix is to store your PAT token as a Databricks secret, and then in your job definition or init script, use that secret to set the environment variable or auth config before installing the package. Since you're using uv for building, make sure it supports pulling credentials from environment variables or a .pypirc file, and set that up using the secret. This way, the job cluster can authenticate properly with Azure Artifacts when it tries to install the private dependency. Let me know if you want a quick example of how to pass the secret into your DAB job! 

Regards,

Brahma

[sandy311]

Hi @Brahmareddy 

Thanks for details, could you please provide examples if possible?