- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-17-2022 03:41 AM
The situation is as following: i have a sheduled job, which uses dbutils.notebook.run(path,timeout) . During the last week everything worked smooth. During the weekend the job began to fail, at the dbutils.notebook.run(path,timeout) command. I get the following error:
com.databricks.WorkflowException: com.databricks.NotebookExecutionException: FAILED: assertion failed: Attempted to set keys (credentials) in the extraContext, but these keys were not in the set of valid keys: {commandResultJsonMaxBytes, displayRowLimitV2, notebook_path, thresholdForStoringInDbfs, enableStoringResultsInDbfs, api_url, aclPathOfAclRoot, displayByteLimitV2}
to be honest, i am not 100% sure if my hoster changed anything in the databricks config during the weekend, but maybe somebody knows which situation would result in such an error.
- Labels:
-
Error
-
Job
-
Jobs & Workflows
-
Strange Error
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-17-2022 04:37 AM
Hi @Mirko Ludewig
This issue seems to be with High Concurrency cluster. Could you please confirm if you are using high concurrency cluster?.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-17-2022 04:37 AM
Hi @Mirko Ludewig
This issue seems to be with High Concurrency cluster. Could you please confirm if you are using high concurrency cluster?.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-17-2022 10:46 PM
Yes this issue faced in high concurrency Cluster when pass through is enabled in Azure data bricks. Any insights about this issue will be helpful?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-17-2022 11:54 PM
@DARSHAN BARGAL @Shaik Maheer first, exsuce me for the delayed answer. I Asked my admin to create a high con cluster for me as well, because i wanted to check both cases. In fact i used a standard cluster all the time. Strangely today everything works fine. With both high con and standaard clusters. I guess that is a good thing
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-18-2022 12:01 AM
Hi @Mirko Ludewig @Shaik Maheer ,
Thank you for the update. We applied the fix from the backend to resolve this issue. We apologies for the inconvenience caused here. I hope everything is working fine now at your end.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-18-2022 06:27 PM
@DARSHAN BARGAL
I am facing a similar error at a client location today (19Jan2022). This is on a pass-thru high con cluster. What was the fix you have applied to resolve this issue?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-31-2022 02:27 PM
Hello Support,
Still i have similar kind of problem.
Error: com.databricks.NotebookExecutionException: FAILED
Cluster Mode : Standard
I tried fetching the data from the event hubs. Previously , it worked good on one databricks instance. with runtime( 7.6 (includes Apache Spark 3.0.1, Scala 2.12)
spark.executor.extraJavaOptions -Dlog4j2.formatMsgNoLookups=true
spark.driver.extraJavaOptions -Dlog4j2.formatMsgNoLookups=true
spark.databricks.delta.preview.enabled true)
Later i tried running on the newly created instance with runtime (7.5 (includes Apache Spark 3.0.1, Scala 2.12)
spark.executor.extraJavaOptions -Dlog4j2.formatMsgNoLookups=true
spark.driver.extraJavaOptions -Dlog4j2.formatMsgNoLookups=true
spark.databricks.delta.preview.enabled true)
Please, help me resolve the issue.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-31-2022 02:30 PM
attached the error file.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-18-2022 12:23 AM
@DARSHAN BARGAL thank you for the explanation. It is good to know that i can recieve help quickly here!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-18-2022 03:32 AM
Hi @DARSHAN BARGAL ,
we have the same problem and it is still present at 11:31 utc
Cluster : standard
Passtrough :on
Runtime : 9.1 ml
region : azure francecentral
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-18-2022 03:37 AM
@Florent POUSSEROT my check was on runtime 10.1
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-01-2022 11:22 PM
Hi @Florent POUSSEROT
Apologies for the delay. Could you please confirm if you are still facing the issue?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-02-2022 12:33 PM
Hi, for the moment, it's ok