Databricks Community

Data_Analytics1 · ‎03-30-2023

Whenever I tried to execute the cell, it is not executing and it says waiting to run and gets stuck there.

I tried this with different clusters but the problem still persist.

What should be the standard process if I create a new cluster and wants to attach it to the newly created job so that job should run seamlessly?

pvignesh92 · ‎03-31-2023

Hi. Did you check the Event log of the cluster to see if the cluster is getting spinned up? Could you confirm if you are using Community Edition or your own cluster for this?

Data_Analytics1 · ‎03-31-2023

Hi @Vigneshraja Palaniraj, I am using my own cluster.

Databricks Runtime Version: 11.3 LTS (includes Apache Spark 3.3.0, Scala 2.12)

Unity Catalog enabled.

Worker type: Standard_DS3_v2, 14 GB Memory, 4 Cores

Workers: 1

Driver type: Standard_DS3_v2, 14 GB Memory, 4 Cores

pvignesh92 · ‎03-31-2023

Please check the event log when you run a notebook and see if the cluster is spinning up or not

Anonymous · ‎03-31-2023

Hi @Mahesh Chahare

Hope everything is going great.

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you.

Cheers!

NandiniN · ‎04-03-2023

Hello @Mahesh Chahare ,

The "Waiting to run" message appears usually when the cluster start or library installation is in progress.

If you are running the commands on an interactive notebook, and you have clicked on "Run All", the commands will get executed sequentially. (Only one command will be running, others will be in the Waiting to run status)

From your comment I understand, you want to run a job and a job cluster should automatically get attached to the job and run, you may use the job scheduler for the notebook.

Please refer to https://docs.databricks.com/workflows/jobs/jobs.html

Hope this helps.

Thanks & Regards,

Nandini