topic Re: Azure databrick throwing 'Py4JJavaError: An error occurred while calling o267._run.' error while calling one notebook from another notebook. Has someone come across such error? Can some one suggest the solution if faced similar issue. in Data Engineering

Azure databrick throwing 'Py4JJavaError: An error occurred while calling o267._run.' error while calling one notebook from another notebook. Has someone come across such error? Can some one suggest the solution if faced similar issue.

Jessy — Tue, 27 Jul 2021 14:10:26 GMT

'Py4JJavaError: An error occurred while calling o267._run.'

Re: Azure databrick throwing 'Py4JJavaError: An error occurred while calling o267._run.' error while calling one notebook from another notebook. Has someone come across such error? Can some one suggest the solution if faced similar issue.

Pierre_MASSE — Sat, 18 Dec 2021 00:24:32 GMT

Hi,

I have the same issue, when launching a notebook from a caller notebook through `dbutils.notebook.run`.

What is weird is that when manually launching the callee notebook, everything goes smoothly. I am also positive that at least one run had been successful even when called by the outer notebook.

What is weird is that when I get to view the inner notebook run, I have a Pandas related exception (KeyError: "None of [Index(['address'], dtype='object')] are in the [columns]"). But I really don't think that it is related to my code as, like mentioned above, the code works when the inner notebook is run directly.

The full visble stack is:

Py4JJavaError: An error occurred while calling o1141._run.

: com.databricks.WorkflowException: com.databricks.NotebookExecutionException: FAILED

at com.databricks.workflow.WorkflowDriver.run(WorkflowDriver.scala:71)

at com.databricks.dbutils_v1.impl.NotebookUtilsImpl.run(NotebookUtilsImpl.scala:122)

at com.databricks.dbutils_v1.impl.NotebookUtilsImpl._run(NotebookUtilsImpl.scala:89)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)

at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:380)

at py4j.Gateway.invoke(Gateway.java:295)

at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)

at py4j.commands.CallCommand.execute(CallCommand.java:79)

at py4j.GatewayConnection.run(GatewayConnection.java:251)

at java.lang.Thread.run(Thread.java:748)

Caused by: com.databricks.NotebookExecutionException: FAILED

at com.databricks.workflow.WorkflowDriver.run0(WorkflowDriver.scala:117)

at com.databricks.workflow.WorkflowDriver.run(WorkflowDriver.scala:66)

... 13 more

Re: Azure databrick throwing 'Py4JJavaError: An error occurred while calling o267._run.' error while calling one notebook from another notebook. Has someone come across such error? Can some one suggest the solution if faced similar issue.

Anonymous — Mon, 20 Dec 2021 20:31:31 GMT

Hello @Pierre MASSE! My name is Piper, and I'm a moderator for Databricks. Thank you for continuing the discussion. Let's see what the community has to say or we'll circle back to you. 😎

Re: Azure databrick throwing 'Py4JJavaError: An error occurred while calling o267._run.' error while calling one notebook from another notebook. Has someone come across such error? Can some one suggest the solution if faced similar issue.

jose_gonzalez — Wed, 26 Jan 2022 00:29:08 GMT

Hi @Pierre MASSE ,

Can you access your child notebook? According to the error message, the error "Caused by: com.databricks.NotebookExecutionException: FAILED" shows that something failed inside of your children notebook.

Re: Azure databrick throwing 'Py4JJavaError: An error occurred while calling o267._run.' error while calling one notebook from another notebook. Has someone come across such error? Can some one suggest the solution if faced similar issue.

Anonymous — Thu, 12 May 2022 13:58:39 GMT

Hey there @Jessy and @Pierre MASSE

Hope you guys are doing great! Thank you for posting your questions.

Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too.

Cheers!

Re: Azure databrick throwing 'Py4JJavaError: An error occurred while calling o267._run.' error while calling one notebook from another notebook. Has someone come across such error? Can some one suggest the solution if faced similar issue.

Pierre_MASSE — Thu, 12 May 2022 15:16:41 GMT

Hi Varika,

I did get an answer from StackOverflow here.

I was working with a python notebook, and the issue I had was that passing a parameter to an inner notebook through `dbutils.notebook.run` and `dbutils.widgets.get` casts all passed parameters to string. It caused an exception in the called notebook.

I will mark @Jose Gonzalez's answer as the best one - see link above to have more details from my specific issue.

Regards,

Pierre

Re: Azure databrick throwing 'Py4JJavaError: An error occurred while calling o267._run.' error while calling one notebook from another notebook. Has someone come across such error? Can some one suggest the solution if faced similar issue.

Anonymous — Fri, 13 May 2022 11:36:02 GMT

Hi @Pierre MASSE

Thank you so much for getting back to us. It's really great of you to send in the solution and mark the answer as best.

We really appreciate your time.

Wish you a great Databricks journey ahead!