topic Re: Unified access token to access Databricks and AWS resourse in Data Engineering

Unified access token to access Databricks and AWS resourse

Abishrp — Tue, 24 Dec 2024 04:54:16 GMT

In azure databricks, it is able to access azure resources and databricks apis using unified MS Entra Id tokens, Similary how can i able to access aws resources and databricks apis (workspace created in cloudformation) using unified access tokens. Here i use sts to access aws resources currently.

Re: Unified access token to access Databricks and AWS resourse

Walter_C — Tue, 24 Dec 2024 12:04:18 GMT

By setting up the IAM role and instance profile, and configuring your Databricks clusters to use the instance profile, you can achieve unified access to AWS resources. For Databricks API access, using OAuth tokens or Databricks PAT tokens ensures secure and managed authentication.

Re: Unified access token to access Databricks and AWS resourse

Abishrp — Tue, 24 Dec 2024 12:14:25 GMT

But there is an option to access APIs using MS Entra ID service principal authentication in Azure and Google Cloud Platform Credential Authentication in GCP,
Why cloud provider-based token management is not available in AWS ?

Re: Unified access token to access Databricks and AWS resourse

Walter_C — Tue, 24 Dec 2024 12:20:26 GMT

I will try to validate on why this is the case, but I have confirmed that indeed for API calls in AWS the token needs to be Databricks generated.