topic Re: "Waiting to run" status of the cell in Data Engineering

"Waiting to run" status of the cell

Data_Analytics1 — Fri, 31 Mar 2023 04:57:26 GMT

Whenever I tried to execute the cell, it is not executing and it says waiting to run and gets stuck there.

I tried this with different clusters but the problem still persist.

What should be the standard process if I create a new cluster and wants to attach it to the newly created job so that job should run seamlessly?

Re: "Waiting to run" status of the cell

pvignesh92 — Fri, 31 Mar 2023 07:41:20 GMT

Hi. Did you check the Event log of the cluster to see if the cluster is getting spinned up? Could you confirm if you are using Community Edition or your own cluster for this?

Re: "Waiting to run" status of the cell

Data_Analytics1 — Fri, 31 Mar 2023 07:55:13 GMT

Hi @Vigneshraja Palaniraj, I am using my own cluster.

Databricks Runtime Version: 11.3 LTS (includes Apache Spark 3.3.0, Scala 2.12)

Unity Catalog enabled.

Worker type: Standard_DS3_v2, 14 GB Memory, 4 Cores

Workers: 1

Driver type: Standard_DS3_v2, 14 GB Memory, 4 Cores

Re: "Waiting to run" status of the cell

pvignesh92 — Fri, 31 Mar 2023 14:53:51 GMT

Please check the event log when you run a notebook and see if the cluster is spinning up or not

Re: "Waiting to run" status of the cell

Anonymous — Sat, 01 Apr 2023 02:08:45 GMT

Hi @Mahesh Chahare

Hope everything is going great.

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you.

Cheers!

Re: "Waiting to run" status of the cell

NandiniN — Mon, 03 Apr 2023 09:05:26 GMT

Hello @Mahesh Chahare ,

The "Waiting to run" message appears usually when the cluster start or library installation is in progress.

If you are running the commands on an interactive notebook, and you have clicked on "Run All", the commands will get executed sequentially. (Only one command will be running, others will be in the Waiting to run status)

From your comment I understand, you want to run a job and a job cluster should automatically get attached to the job and run, you may use the job scheduler for the notebook.

Please refer to https://docs.databricks.com/workflows/jobs/jobs.html

Hope this helps.

Thanks & Regards,

Nandini