Internal error: Attach your notebook to a different compute or restart the current compute.

aayusha3 — Sun, 19 Nov 2023 16:43:13 GMT

I am currently using a personal computer cluster [13.3 LTS (includes Apache Spark 3.4.1, Scala 2.12)] on GCP attached to a notebook. After running a few command lines without an issue, I end up getting this error

Internal error. Attach your notebook to a different compute or restart the current compute. java.lang.RuntimeException: abort: DriverClient destroyed at com.databricks.backend.daemon.driver.DriverClient.$anonfun$poll$3(DriverClient.scala:577) at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307) at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:54) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:77) at com.databricks.threading.DatabricksExecutionContext$InstrumentedRunnable.run(DatabricksExecutionContext.scala:36) at com.databricks.threading.NamedExecutor$$anon$2.$anonfun$run$2(NamedExecutor.scala:366) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:420) at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:418) at com.databricks.threading.NamedExecutor.withAttributionContext(NamedExecutor.scala:285) at com.databricks.threading.NamedExecutor$$anon$2.$anonfun$run$1(NamedExecutor.scala:364) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at com.databricks.context.integrity.IntegrityCheckContext$ThreadLocalStorage$.withValue(IntegrityCheckContext.scala:44) at com.databricks.threading.NamedExecutor$$anon$2.run(NamedExecutor.scala:356) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829)

Within the databricks UI, I get the following errors:

When I restart my cluster, I am able to run queries without any errors 10-15 times but then I end up getting this same error again. I tried running my queries with a new cluster but i face the same errors after 10-15 runs.

This seems to be a user isolated error since a colleague tried using his own personal compute cluster and was able to run queries without any issues.

Here are the things I have tried so far while logged into my accounting:

Creating a completely new cluster -- Still having issues after a couple of queries
Restarting my existing cluster -- Still having issues after a couple of queries
Open a new incognito tab and run queries -- Still get the same errors

Only thing left for me to test is try logging into Databricks and running my queries on a separate device.

Please let me know if anyone else has faced this issue previously and if there is any way to resolve this.

Thanks,

Re: Internal error: Attach your notebook to a different compute or restart the current compute.

amandaolens — Mon, 20 Nov 2023 09:23:42 GMT

i am also facing the same issue.

Re: Internal error: Attach your notebook to a different compute or restart the current compute.

amandaolens — Mon, 20 Nov 2023 09:25:40 GMT

@aayusha3 have you found any fix for this issue?

Re: Internal error: Attach your notebook to a different compute or restart the current compute.

Martin74 — Wed, 22 Nov 2023 09:11:03 GMT

We have facing the same issue in the last days.
When testing it again today, it seems solved.
Anybody has the same experience?

Re: Internal error: Attach your notebook to a different compute or restart the current compute.

amandaolens — Wed, 22 Nov 2023 09:18:03 GMT

@Martin74 same here martin.

topic Internal error: Attach your notebook to a different compute or restart the current compute. in Administration & Architecture

Internal error: Attach your notebook to a different compute or restart the current compute.

Re: Internal error: Attach your notebook to a different compute or restart the current compute.

Re: Internal error: Attach your notebook to a different compute or restart the current compute.

Re: Internal error: Attach your notebook to a different compute or restart the current compute.

Re: Internal error: Attach your notebook to a different compute or restart the current compute.