topic Re: Databricks clusters stuck on Pending and Terminating state indefinitely in Data Engineering

Databricks clusters stuck on Pending and Terminating state indefinitely

antoooks — Fri, 29 Oct 2021 06:15:48 GMT

Hi everyone,

Our company is using Databricks on GKE. It works fine until suddenly when we try to create and terminate clusters today, it got stuck on Pending and Terminating state for hours (now more than 6 hours). There is no conclusion can be drawn from Databricks clusters API and Kubernetes, except that it gives "Finding instances for new nodes, acquiring more instances if necessary". I tried restarting all Databricks pods on Kubernetes but it's still happening. Can someone help us about this? Please find screenshot below for more context.

Regards,

Kurnianto

Re: Databricks clusters stuck on Pending and Terminating state indefinitely

antoooks — Fri, 29 Oct 2021 07:40:47 GMT

We have figured it out. So we have to allow 0.0.0.0:443 to our Firewall rules, it's not the best outcome out there since we are using private GKE clusters. However it solves our issue.

Thanks.

Re: Databricks clusters stuck on Pending and Terminating state indefinitely

jose_gonzalez — Fri, 29 Oct 2021 23:02:03 GMT

Hi @Kurnianto Trilaksono Sutjipto ,

Thanks for the follow-up. Could you mark your solution as best, so others can quickly find it?

Re: Databricks clusters stuck on Pending and Terminating state indefinitely

Mostafa — Sun, 17 Apr 2022 12:28:52 GMT

Hi everyone!

I have similar issues and I don't know what I should do. please help me out. I am a student and I need this to be fixed to move forward.

It has been a few hours since DB tries to delete those clusters. When I create a new one, it gets stuck in the pending mode forever!

Thank you so much in advance! I am using the community edition.

Re: Databricks clusters stuck on Pending and Terminating state indefinitely

gazzyjuruj — Sun, 17 Apr 2022 14:53:29 GMT

same, it's happening today since morning.

Re: Databricks clusters stuck on Pending and Terminating state indefinitely

bhawana-pandey — Thu, 19 Jan 2023 07:51:55 GMT

Hi @Kurnianto Trilaksono Sutjipto : Figured out after multiple connects that This is typically a cloud provider issue. You can file a support ticket if the issue persists.