I'm coming back to provide an updated solution that doesn't rely on the implementation detail of the user name (e.g., libraries) - which is not considered a contract and could potentially change and break in the future.

The key is to use the --global flag when calling pip config set. Unfortunately, aws codeartifact login doesn't do this even though it does set global.index-url which is namespaced, but only for --user. The workaround is to use aws codeartifact get-authorization-token instead of login and then manually construct the index url, passing in the credentials with the token:

#!/bin/bash
set -e

echo "Authenticating to AWS Code Artifact"
CODEARTIFACT_AUTH_TOKEN=$(aws codeartifact get-authorization-token --domain <code-artifact-domain> --region <region of domain> --output text --query authorizationToken)
/databricks/python3/bin/python -m pip config --global set global.index-url https://aws:${CODEARTIFACT_AUTH_TOKEN}@<codeartifact repo url>

echo "Adding extra index to pip config"
/databricks/python3/bin/python -m pip config --global set global.extra-index-url https://pypi.org/simple