Unexpected failure while fetching notebook - What can we do from our side?

[killjoy]

Hello!

We got some pipelines running in Azure Data Factory that call Databricks Notebooks to run data transformations. This morning at 6:21 AM (UTC) we got an error " Unexpected failure while fetching notebook" inside a notebook that calls another one.

Upon checking the Driver Logs for that job in particular, in the "Standard Error" it appears:

OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
ANTLR Tool version 4.7 used for code generation does not match the current runtime version 4.8

And then some lines below the initial error

Py4JJavaError: An error occurred while calling o1771._run.
: com.databricks.WorkflowException: com.databricks.NotebookExecutionException: FAILED: Unexpected failure while fetching notebook.

The work around I got was to rerun the pipeline and it succeeded but I'm still not sure what happened, if it's something from our side, is there something I can do to prevent this from happening at again?

[AmanGill]

Hello, we are in Western Canada running on Azure databricks. We had the exact same issue which started at 6:21 AM (UTC) and resolved on it's own at 6:44 AM (UTC) on 11/04/2023.

[Anonymous]

@Rita Fernandes​ :

Based on the error message you provided, it seems like the issue might be related to the version mismatch between the ANTLR tool used for code generation and the current runtime version. Additionally, the error message suggests that the failure occurred while fetching the notebook, which could be due to a variety of factors such as network connectivity issues or server availability problems.

Does it work now?