topic dbutils.notebook.run raises HTTP 401 Unauthorized Error in Data Engineering

dbutils.notebook.run raises HTTP 401 Unauthorized Error

Markus — Thu, 20 Oct 2022 22:47:22 GMT

Hello,

since a while I use dbutils.notebook.run for multiple calling of additional notebooks and passing parameters to them. So far I could use the function without any difficulties - also today.

But since a few hours now I get the following error message when calling the notebook:

com.databricks.WorkflowException: com.databricks.common.client.UnexpectedHttpError: HTTP request failed with status: HTTP/1.1 401 Unauthorized

The call (even this simplified test call generates the error):

test = dbutils.notebook.run("./test2", timeout_seconds = 300 )

Does anyone know how this error occurs and how I can fix it?

I tried it on different clusters, terminated/restarted the clusters and have still the same problem. Could an authorization/token have expired between the last successful run and the error?

Best regards

Markus

Re: dbutils.notebook.run raises HTTP 401 Unauthorized Error

Markus — Mon, 24 Oct 2022 09:45:10 GMT

Hello Community,

the issue occurred due to a changed central configuration.

Recommendation by Databricks: "Admin Protection: New feature and security recommendations for No Isolation Shared clusters"

Here is the link to the current restrictions: Enable admin protection for “No isolation shared” clusters on your account | Databricks on Google Cloud

Best regards

Markus

Re: dbutils.notebook.run raises HTTP 401 Unauthorized Error

Debayan — Tue, 25 Oct 2022 07:35:32 GMT

Hi @Markus F. , Could you please paste the whole error snippet here? You could open a databricks support case too with the whole error code.