Is there a way to switch default cluster associated with a workflow job
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-13-2025 08:18 PM - edited 02-13-2025 08:19 PM
Hi,
I have a workflow job that is connected to a default cluster (see blow)
I know I can swap the cluster.
However, sometimes the cluster is not active but when I start the workflow job, I will wait for the cluster to beome activated. It will take some time.
Is there a way that if the default cluster is not active when I run the job, another active cluster will be connected and to run the job? How to automate this? Through an API or any way?
Any ideas are welcome.
Thanks in advance!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-14-2025 05:20 AM
So you run jobs on interactive clusters? Don't do that.
Use job clusters, way cheaper. However they need spin-up time. If you do not want that, look at Serverless Job Clusters!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-17-2025 09:29 AM
I'm using a Databricks App, and the workflow jobs are inside running the app.
Do you still recommend using job clusters for app?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-18-2025 04:01 AM
I suppose you can call the databricks api to run those workflows?
Or is that a no go?

