topic Re: Azure AAD token with Databricks for User defined managed Identity inside Docker Container in Data Engineering

Azure AAD token with Databricks for User defined managed Identity inside Docker Container

dsura — Mon, 24 Oct 2022 14:56:33 GMT

Hi, We are currently using a Azure AAD Token inorder to authenticate with Databricks instead of generating Personal Access Tokens from Databricks. We have a multi-tenant architecture and so we are using Azure container instances to run multiple transformation pipelines parallel using dbT.

Inorder to authenticate with databricks we generate an AAD token inside the ACI using the user defined managed identity (UDMI) which has contributor and reader rights on databricks. We do get the AAD token back successfully. However, the token when passed to databricks returns a 403 error.

UDMI is also an admin in the databricks workspace.

[0m18:11:15.979533 [debug] [MainThread]: Opening a new connection, currently in state init
[0m18:11:18.694997 [debug] [MainThread]: Databricks adapter: failed to connect: Error during request to server: : User not authorized.
[0m18:11:18.694997 [debug] [MainThread]: Databricks adapter: <class 'databricks.sql.exc.RequestError'>: Error during request to server: : User not authorized.
[0m18:11:18.694997 [debug] [MainThread]: Databricks adapter: attempt: 1/30
[0m18:11:18.695990 [debug] [MainThread]: Databricks adapter: bounded-retry-delay: None
[0m18:11:18.695990 [debug] [MainThread]: Databricks adapter: elapsed-seconds: 2.699450731277466/900.0
[0m18:11:18.695990 [debug] [MainThread]: Databricks adapter: error-message: : User not authorized.
[0m18:11:18.695990 [debug] [MainThread]: Databricks adapter: http-code: 403

Is there anything that we are missing and have to add more to this? Any help on this is appreciated.

Re: Azure AAD token with Databricks for User defined managed Identity inside Docker Container

dsura — Tue, 25 Oct 2022 15:38:14 GMT

Hi @Debayan Mukherjee , We are using the below DBR. Can you let me know if this is good? Or I need to use 11.3?

az login --identity --username /subscriptions/{subscription_ID}/resourcegroups/{resource_Group}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{user_defined_managed_Identity}
 
aad_token=$(az account get-access-token --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d --query "accessToken" --output tsv)

Re: Azure AAD token with Databricks for User defined managed Identity inside Docker Container

dsura — Wed, 26 Oct 2022 15:35:26 GMT

@Debayan Mukherjee , I am still getting the above error. I tried with 11.2, 11.3 LTS.

Can you let me know what permissions did you give to the UDMI in your poc? Or schedule a call to discuss about a possible solution?

Re: Azure AAD token with Databricks for User defined managed Identity inside Docker Container

dsura — Wed, 26 Oct 2022 20:30:30 GMT

@Debayan Mukherjee , We were able to fix the issue by changing the few guids that we used. Apparently the managed identity application id wasnt getting deployed appropriately by Terraform which caused this issue. It wasn't an issue related to Databricks Runtime.

Re: Azure AAD token with Databricks for User defined managed Identity inside Docker Container

Debayan — Tue, 25 Oct 2022 05:51:05 GMT

Hi, This looks like an old issue which we have faced earlier, Could you please update the DBR to the latest version and try again?

Re: Azure AAD token with Databricks for User defined managed Identity inside Docker Container

Debayan — Wed, 26 Oct 2022 07:17:17 GMT

Hi @Dharit Sura , Could you please try DBR 9.1 LTS , 11.2 and 11.3 LTS?