Hi!
I'm exploring options for workload identity federation and have a question about cross-cloud scenarios.
Current Setup:
- Azure Databricks workspace
- Workloads running in GCP (planning to use GKE/Kubernetes)
- Need to authenticate GCP-based workloads to Azure Databricks APIs without managing secrets
Question: Is the Databricks federation policy designed to support cross-cloud federation? Specifically, can I configure a service principal federation policy in Azure Databricks to accept tokens from a GCP Kubernetes cluster?
Looking at the documentation, I see Kubernetes is listed as a supported identity provider with this example configuration:
Issuer: https://kubernetes.default.svc
Audience: https://kubernetes.default.svc
Subject: system:serviceaccount:namespace:podname
My specific concerns:
- Would this work with a GKE cluster's external issuer URL instead of the internal kubernetes.default.svc?
- Are there any known limitations or considerations for cross-cloud federation scenarios?
- Has anyone successfully implemented GCP workload identity → Azure Databricks authentication?
Alternative considered: I'm aware I could potentially use Azure Entra ID as an intermediary, but I'm hoping to establish direct federation if possible to reduce complexity.
Any insights or experiences with cross-cloud federation would be greatly appreciated!
Thanks!