Use OAuth based on a user or service principal which will respect the given access granted through unity catalog to them:https://docs.databricks.com/aws/en/dev-tools/auth/oauth-u2m#gsc.tab=0Whereas a PAT is only for use to the specified user it was g...
A solution that worked, in addition to having the HTTP_PROXY and HTTPS_PROXY variables set globally, was to add the following definition to the compute policy:"spark_env_vars.NO_PROXY": { "type": "fixed", "value": "localhost,127.0.0.1,169.254.1...
Bumping this as I am having the same issue.Is the solution to just not define the proxy vars globally?Is there something to add in the NO_PROXY or the spark_conf to allow for communication within databricks between storage accounts to not go through ...
