cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Can Databricks federation policy support cross-cloud authentication?

Direo
Contributor II

Hi!

I'm exploring options for workload identity federation and have a question about cross-cloud scenarios.

Current Setup:

  • Azure Databricks workspace
  • Workloads running in GCP (planning to use GKE/Kubernetes)
  • Need to authenticate GCP-based workloads to Azure Databricks APIs without managing secrets

Question: Is the Databricks federation policy designed to support cross-cloud federation? Specifically, can I configure a service principal federation policy in Azure Databricks to accept tokens from a GCP Kubernetes cluster?

Looking at the documentation, I see Kubernetes is listed as a supported identity provider with this example configuration:
Issuer: https://kubernetes.default.svc
Audience: https://kubernetes.default.svc
Subject: system:serviceaccount:namespace:podname

My specific concerns:

  1. Would this work with a GKE cluster's external issuer URL instead of the internal kubernetes.default.svc?
  2. Are there any known limitations or considerations for cross-cloud federation scenarios?
  3. Has anyone successfully implemented GCP workload identity → Azure Databricks authentication?

Alternative considered: I'm aware I could potentially use Azure Entra ID as an intermediary, but I'm hoping to establish direct federation if possible to reduce complexity.

Any insights or experiences with cross-cloud federation would be greatly appreciated!

Thanks!

0 REPLIES 0

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now