Unable to connect to any cluster from a notebook

[RPalmer]

I'm experiencing an unusual issue following my return from annual leave. I'm unable to connect to any compute from a notebook (both Classic Compute and Serverless) this is despite having Can Manage permissions on the clusters.

The error shown is: "Unknown cluster: Insufficient cluster permissions or cluster doesn't exist anymore"

Notably I can still do all of the following without issue:

• Browse and preview data in Unity Catalog
• Run queries in the SQL Editor against our SQL Warehouse
• Create, manage, and start clusters

Steps already taken with no resolution:

• Full laptop reboot
• Named access with Can Manage granted on a specific cluster
• Checked browser/session is not stale by logging out of Databricks and back in

We are on Azure Databricks. Has anyone seen this pattern before where SQL Warehouse and Unity Catalog browsing work fine but notebook compute attachment is broken for a specific user?
The cluster does start when I choose it and shows the green dot but then looks like the attached screenshot and the only option given is to detach.

[juan_maedo]

Hi,

I haven’t seen this exact issue before with a notebook, but regarding the fact that you’ve been away for a while: is that notebook old, or was it created recently?
I’m wondering whether, if it’s an old notebook, it might be looking for some of the cluster IDs it had back then and has them stored, and now it can’t find them because the cluster was updated or new clusters were created.

I’d try creating a new notebook from both your workspace and from the shared workspace, just to rule things out and narrow down whether they still don’t work.

I’ll wait for more info!

[RPalmer]

Hmm. Despite showing the same error in the cluster dropdown if I run a select in the notebook I had already started working on before my leave and in fresh notebook, it successfully runs select queries.

[juan_maedo]

The only thing I can think of is that it may still be storing the id of cluster “AAA” from before your year off, and it keeps associating it because of the cluster name. In other words, you could have:

• Cluster AAA (old) - ID: XXXXXX - Runtime version 15.2

• Cluster AAA (new) - ID: BBBBB - Runtime version 16.4

Your notebook may still have the old ID XXXXX saved, but since it matches the name AAA, it triggers that error. The other option would be to do a reattach with Serverless and try again, or restart the cluster that is currently conflicting.

Also, restarting the notebook session.

Let us know!

[na4]

I am also having this problem with both serverless and classic. I'm not coming off any sort of leave. Is this a bigger issue?

[MalachiBunn]

We've had at least two people report this issue starting this morning, it's not consistent behavior, many users are not seeing it. 

[mcharlONT]

Also having this issue with my notebooks today

[alex1234]

Im also having the same issue

[RPalmer]

Good news. When I signed into Databricks this morning the issue seems to have gone away. All I did yesterday was sign out of databricks at the end of the workday and then sign in again this morning. So perhaps the issue was in Databricks for a subset of users and it has now been resolved.

[Advika · Accepted solution]

Hi all, the issue should now be mitigated. Really appreciate your patience on this!
Do let us know if you’re still experiencing any problems.