cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Is there a way to switch default cluster associated with a workflow job

ianchenmu
New Contributor III

Hi, 

I have a workflow job that is connected to a default cluster (see blow)

ianchenmu_0-1739506495579.png

I know I can swap the cluster. 

However, sometimes the cluster is not active but when I start the workflow job, I will wait for the cluster to beome activated. It will take some time. 

Is there a way that if the default cluster is not active when I run the job, another active cluster will be connected and to run the job? How to automate this? Through an API or any way?

Any ideas are welcome.

Thanks in advance!

 

3 REPLIES 3

-werners-
Esteemed Contributor III

So you run jobs on interactive clusters?  Don't do that.
Use job clusters, way cheaper.  However they need spin-up time.  If you do not want that, look at Serverless Job Clusters!

ianchenmu
New Contributor III

I'm using a Databricks App, and the workflow jobs are inside running the app.

Do you still recommend using job clusters for app?

-werners-
Esteemed Contributor III

I suppose you can call the databricks api to run those workflows?
Or is that a no go?

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now