Can Databricks Jobs Run on Kubernetes Clusters?

ASH1243434 — Tue, 23 Jun 2026 11:33:54 GMT

Context: We're exploring using Kubernetes (EKS) as our compute infrastructure instead of Databricks managed clusters. We want to understand if Databricks can orchestrate, deploy, and monitor jobs that run on a Kubernetes cluster.

Questions:

Is it possible to configure Databricks Jobs to run workloads on an external Kubernetes cluster?
Can Databricks manage the job lifecycle (submit, monitor, track logs) on Kubernetes pods?
What's the recommended way to use Databricks as an orchestration layer for Kubernetes workloads?

What we want to do:

Write Databricks jobs/notebooks
Have those jobs execute as pods in our Kubernetes cluster (not Databricks-managed clusters)
Monitor job execution and logs from the Databricks UI
Keep using Databricks features (MLflow, Delta Lake) for data and model management

Why this is important:

Scalability: Kubernetes provides better horizontal scaling for large workloads
Faster cluster initialization: Pre-running EKS cluster eliminates spin-up delays
Cost efficiency: EKS with spot instances is more cost-effective than always-on Databricks clusters
Multi-tenant support: Share compute infrastructure across different teams and workloads

Re: Can Databricks Jobs Run on Kubernetes Clusters?

szymon_dybczak — Tue, 23 Jun 2026 13:42:44 GMT

Hi @ASH1243434 ,

Unfortunately, the Databricks cannot natively route job execution into your EKS cluster. There is no "external compute" or "bring your own Kubernetes" option in Databricks Jobs configuration.

If my answer was helpful, please consider marking it as accepted solution

topic Can Databricks Jobs Run on Kubernetes Clusters? in Machine Learning

Can Databricks Jobs Run on Kubernetes Clusters?

Re: Can Databricks Jobs Run on Kubernetes Clusters?