Azure databrick throwing 'Py4JJavaError: An error occurred while calling o267._run.' error while calling one notebook from another notebook. Has someone come across such error? Can some one suggest the solution if faced similar issue.

[Jessy]

'Py4JJavaError: An error occurred while calling o267._run.'

[Pierre_MASSE]

Hi,

I have the same issue, when launching a notebook from a caller notebook through `dbutils.notebook.run`.

What is weird is that when manually launching the callee notebook, everything goes smoothly. I am also positive that at least one run had been successful even when called by the outer notebook.

What is weird is that when I get to view the inner notebook run, I have a Pandas related exception (KeyError: "None of [Index(['address'], dtype='object')] are in the [columns]"). But I really don't think that it is related to my code as, like mentioned above, the code works when the inner notebook is run directly.

The full visble stack is:

Py4JJavaError: An error occurred while calling o1141._run.

: com.databricks.WorkflowException: com.databricks.NotebookExecutionException: FAILED

at com.databricks.workflow.WorkflowDriver.run(WorkflowDriver.scala:71)

at com.databricks.dbutils_v1.impl.NotebookUtilsImpl.run(NotebookUtilsImpl.scala:122)

at com.databricks.dbutils_v1.impl.NotebookUtilsImpl._run(NotebookUtilsImpl.scala:89)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)

at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:380)

at py4j.Gateway.invoke(Gateway.java:295)

at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)

at py4j.commands.CallCommand.execute(CallCommand.java:79)

at py4j.GatewayConnection.run(GatewayConnection.java:251)

at java.lang.Thread.run(Thread.java:748)

Caused by: com.databricks.NotebookExecutionException: FAILED

at com.databricks.workflow.WorkflowDriver.run0(WorkflowDriver.scala:117)

at com.databricks.workflow.WorkflowDriver.run(WorkflowDriver.scala:66)

... 13 more

[Anonymous]

Hello @Pierre MASSE​! My name is Piper, and I'm a moderator for Databricks. Thank you for continuing the discussion. Let's see what the community has to say or we'll circle back to you. 😎

[jose_gonzalez · Accepted solution]

Hi @Pierre MASSE​ ,

Can you access your child notebook? According to the error message, the error "Caused by: com.databricks.NotebookExecutionException: FAILED" shows that something failed inside of your children notebook.

[Anonymous]

Hey there @Jessy​ and @Pierre MASSE​ 

Hope you guys are doing great! Thank you for posting your questions.

Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too.

Cheers!

[Pierre_MASSE]

Hi Varika,

I did get an answer from StackOverflow here.

I was working with a python notebook, and the issue I had was that passing a parameter to an inner notebook through `dbutils.notebook.run` and `dbutils.widgets.get` casts all passed parameters to string. It caused an exception in the called notebook.

I will mark @Jose Gonzalez​'s answer as the best one - see link above to have more details from my specific issue.

Regards,

Pierre

[Anonymous]

Hi @Pierre MASSE​ 

Thank you so much for getting back to us. It's really great of you to send in the solution and mark the answer as best.

We really appreciate your time.

Wish you a great Databricks journey ahead!