topic Internal error. Attach your notebook to a different cluster or restart the current cluster. in Data Engineering

Internal error. Attach your notebook to a different cluster or restart the current cluster.

210573 — Sun, 11 Sep 2022 15:35:16 GMT

Started getting this error while running all the scripts. All the scripts were running fine before. I tried de-attaching and also restart nothing seems to work.

Internal error. Attach your notebook to a different cluster or restart the current cluster.

java.lang.RuntimeException: abort: DriverClient destroyed

at com.databricks.backend.daemon.driver.DriverClient.$anonfun$poll$3(DriverClient.scala:591)

at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)

at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)

at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)

at com.databricks.threading.NamedExecutor$$anon$2.$anonfun$run$1(NamedExecutor.scala:360)

at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)

at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext$1(UsageLogging.scala:398)

at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)

at com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:108)

at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:396)

at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:393)

at com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:287)

at com.databricks.threading.NamedExecutor$$anon$2.run(NamedExecutor.scala:359)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

at java.base/java.lang.Thread.run(Thread.java:834)

Re: Internal error. Attach your notebook to a different cluster or restart the current cluster.

Prabakar — Tue, 13 Sep 2022 04:21:55 GMT

@cloud user This looks like a driver issue or the control plane issue. What is the cluster configuration? Have you tried changing the instance type? From when this started and which region is your workspace deployed?

Re: Internal error. Attach your notebook to a different cluster or restart the current cluster.

Anonymous — Sat, 24 Sep 2022 08:38:27 GMT

Hi @cloud user

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: Internal error. Attach your notebook to a different cluster or restart the current cluster.

tieu_quyen — Wed, 18 Jan 2023 09:46:55 GMT

Hi @210573 (Customer) ,

I got the same error, tried to restart and create a new cluster but the solution does not work. What I did to fix the issue:

Instead of putting in function, break the code out to run line by line. I just want to see where the error happens
Then, I figure out the specific line code in between got an error "Internal error. Attach your notebook to a different cluster or restart the current cluster."
Check that specific codes and fix errors (for my case is about incorrect schema defined even Databrick does not mention exact error of schema). After the schema is fixed, the error is gone

So I think it is more about the coding errors

All the best

Re: Internal error. Attach your notebook to a different cluster or restart the current cluster.

mnzhu — Wed, 22 Nov 2023 10:46:44 GMT

I got the same error, and it turned out to be the schema error too. Thanks!