โ10-24-2022 07:56 AM
Hi, We are currently using a Azure AAD Token inorder to authenticate with Databricks instead of generating Personal Access Tokens from Databricks. We have a multi-tenant architecture and so we are using Azure container instances to run multiple transformation pipelines parallel using dbT.
Inorder to authenticate with databricks we generate an AAD token inside the ACI using the user defined managed identity (UDMI) which has contributor and reader rights on databricks. We do get the AAD token back successfully. However, the token when passed to databricks returns a 403 error.
UDMI is also an admin in the databricks workspace.
[0m18:11:15.979533 [debug] [MainThread]: Opening a new connection, currently in state init
[0m18:11:18.694997 [debug] [MainThread]: Databricks adapter: failed to connect: Error during request to server: : User not authorized.
[0m18:11:18.694997 [debug] [MainThread]: Databricks adapter: <class 'databricks.sql.exc.RequestError'>: Error during request to server: : User not authorized.
[0m18:11:18.694997 [debug] [MainThread]: Databricks adapter: attempt: 1/30
[0m18:11:18.695990 [debug] [MainThread]: Databricks adapter: bounded-retry-delay: None
[0m18:11:18.695990 [debug] [MainThread]: Databricks adapter: elapsed-seconds: 2.699450731277466/900.0
[0m18:11:18.695990 [debug] [MainThread]: Databricks adapter: error-message: : User not authorized.
[0m18:11:18.695990 [debug] [MainThread]: Databricks adapter: http-code: 403
Is there anything that we are missing and have to add more to this? Any help on this is appreciated.
โ10-26-2022 01:30 PM
@Debayan Mukherjeeโ , We were able to fix the issue by changing the few guids that we used. Apparently the managed identity application id wasnt getting deployed appropriately by Terraform which caused this issue. It wasn't an issue related to Databricks Runtime.
โ10-24-2022 10:51 PM
Hi, This looks like an old issue which we have faced earlier, Could you please update the DBR to the latest version and try again?
โ10-25-2022 08:38 AM
Hi @Debayan Mukherjeeโ , We are using the below DBR. Can you let me know if this is good? Or I need to use 11.3?
az login --identity --username /subscriptions/{subscription_ID}/resourcegroups/{resource_Group}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{user_defined_managed_Identity}
aad_token=$(az account get-access-token --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d --query "accessToken" --output tsv)
โ10-26-2022 12:17 AM
Hi @Dharit Suraโ , Could you please try DBR 9.1 LTS , 11.2 and 11.3 LTS?
โ10-26-2022 08:35 AM
โ10-26-2022 01:30 PM
@Debayan Mukherjeeโ , We were able to fix the issue by changing the few guids that we used. Apparently the managed identity application id wasnt getting deployed appropriately by Terraform which caused this issue. It wasn't an issue related to Databricks Runtime.
โ10-26-2022 01:32 PM
Hi @Dharit Suraโ, Thank you for the response. Would you select your answer as the best for the community?
โ10-25-2022 02:22 AM
Hi @Dharit Suraโ โ, We havenโt heard from you since the last response from @Debayan Mukherjeeโ , and I was checking back to see if you have a resolution yet.
If you have any solution, please share it with the community as it can be helpful to others. Otherwise, we will respond with more details and try to help.
Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.