Code on cluster runs idefinitely

darek554 — Sun, 28 Jun 2026 08:51:46 GMT

Hello.

Ive created a custom cluster - m4.large. When i try to execute some code in this cluster the behaviour is as follows:

- Cluster starts, have running status

- I run code, for example print("Hello")

- Code runs indefinitely

- I click interrupt, it stops after long time

- When code is stopped, next code execution runs fine

But this is about python, SQL dont run at all, even SELECT 1; query.

Can someone tell me what can be an issue?

Re: Code on cluster runs idefinitely

Yogasathyandrun — Sun, 28 Jun 2026 09:53:24 GMT

The fact that print("Hello") eventually works but SELECT 1 never completes suggests the cluster may be running but not fully initialized for Spark workloads.

A few things I’d check first:

Cluster Event Log for any provisioning or startup errors.
Spark UI → Executors to confirm workers/executors are actually coming up.
Driver logs for startup exceptions or connectivity issues.
Whether this is a single-node cluster or a cluster with separate workers.

One other thing that stands out is the use of m4.large, which is a fairly old instance family. If possible, try spinning up a small cluster on a newer instance type (for example m5 or m6 generation) and see if the behavior changes.

Also, which Databricks Runtime version are you running, and does the issue occur immediately after cluster startup or only after the cluster has been idle for some time?

topic Code on cluster runs idefinitely in Data Engineering

Code on cluster runs idefinitely

Re: Code on cluster runs idefinitely