Hi @pablobd, Hi, thank you for your question. I’m happy to help you with connecting your AWS CodeArtifact repo to Databricks. 😊
There are a few steps you need to follow to achieve this:
- First, you need to create an HTTPS Git credential in AWS CodeCommit that allows access to your private repo. You can do this by following the instructions in the AWS CodeCommit documentation. The associated IAM user must have “read” and “write” permissions for the repository. You also need to record the password, as you will enter it in Databricks later.
- Second, you need to configure a remote repo in Databricks that points to your private repo using the HTTPS Git credential. You can do this by following the instructions in the Databricks documentation. You will need to enter the repository URL, which should look something like this: https://<aws-account-id>.dkr.ecr.<region>.amazonaws.com/<repository-name>. You will also need to enter the username and password that you created in AWS CodeCommit.
- Third, you need to install the packages from your private repo using pip or another package manager. You can do this by adding a line like this before installing the dependencies: pip config set site.index-url https://<aws-account-id>:<access-token>@<region>-111122223333.d.codeartifact.region.amazonaws.com/pypi/my_repo/simple/. The <aws-account-id> is your AWS account ID, which you can find in the IAM console. The <access-token> is the token that you generated when creating the HTTPS Git credential in AWS CodeCommit. The <region> is your AWS region.
I hope this helps you with connecting your AWS CodeArtifact repo to Databricks. If you have any further questions, please feel free to ask me. 😊
