Need to create an Identity Federation between my Databricks workspace/account and my AWS account

[axelboursin]

Hello,

I need to set up an identification between my Databricks workspace/account and my AWS account, where Databricks is already deployed.

The goal is to make an easy authentification without access and secret keys.

So I thought that OIDC will be the solution, but sadly, it's not clear on the documentation.
Everything on OIDC is about SSO, that's not really what I want.

What is the OIDC url to use to create a federated identity on AWS, to easily assume a role and get some privileges?

Thanks for helping me!

[mark_ott]

To set up identification between your Databricks workspace/account and your AWS account without using access or secret keys, you can leverage OIDC (OpenID Connect) federation. Instead of traditional SSO, what you’re looking for is a model where AWS trusts your Databricks-issued identities such that Databricks users or workloads can assume roles in your AWS account using an OIDC provider and receive temporary AWS credentials.

OIDC Identity Provider URL for AWS Federation

For AWS to trust Databricks identities via OIDC, AWS needs the issuer URL (OIDC discovery endpoint) from Databricks. The URL structure depends on whether your Databricks deployment uses unified login and if you're configuring at workspace level or account level:

• If using workspace-level OIDC and unified login is enabled:

  • https://<your-workspace-hostname>/oidc/.well-known/openid-configuration

    • For example: https://abc-def.cloud.databricks.com/oidc/.well-known/openid-configuration

• If not using unified login or configuring at the Databricks account level:

  • https://accounts.cloud.databricks.com/oidc/accounts/<your-databricks-account-id>/.well-known/openid-configuration

You can find your Databricks account ID in your Databricks admin console. Use the appropriate URL as the "Provider URL" when setting up the OIDC IdP in AWS IAM.​

Steps to Federate AWS IAM with Databricks OIDC

1. Get Databricks OIDC URL: Identify which URL (workspace vs. account) fits your setup.

2. Register OIDC IdP in AWS: In AWS IAM, create a new "Identity Provider" of type "OIDC." Use the Databricks OIDC discovery URL above as the provider URL.

3. Configure IAM Role Trust Policy: In the trust policy for the role you want Databricks workloads/users to assume, specify the newly created OIDC provider as the principal and define the appropriate audience and conditions.

4. Assume Role from Databricks: Use Databricks features/applications (such as Unity Catalog, cluster configuration, or relevant APIs) that support OIDC-based AWS role assumption.

This approach avoids exposing static access or secret keys and leverages short-lived, federated authentication.​

Practical Example

If your workspace is mycompany.cloud.databricks.com and your account ID is 123456789012, the URL for AWS would be:

• https://mycompany.cloud.databricks.com/oidc/.well-known/openid-configuration
  or (account level):

• https://accounts.cloud.databricks.com/oidc/accounts/123456789012/.well-known/openid-configuration

These discovery endpoints allow AWS to validate tokens issued by Databricks for federated role assumption.​

References

• OIDC metadata URI and typical setup practices:​

• Official Databricks documentation for OIDC/OAuth federation:​

If you follow these steps and use the correct OIDC URL structure for your deployment, AWS will be able to trust identities from your Databricks workspace via OIDC federation, letting you assign privileges in a secure, keyless way.