%run notebook fails in Job mode with Py4JJavaError (None.get), but works in interactive notebook

Akshay_Petkar
Valued Contributor

 

Hi everyone,

I’m facing an issue when executing a Databricks job where my notebook uses %run to include other notebooks. I have a final notebook added as a task in a job, and inside that notebook I use %run to call another notebook that contains all my functions. That second notebook again calls another notebook using %run.

When I run the notebook manually in the UI, everything works perfectly. But when the same notebook runs as a job, it fails with a Py4JJavaError showing “java.util.NoSuchElementException: None.get”.

There are no missing widgets or paths because the exact same setup runs fine interactively.

Has anyone else seen this behavior where %run works fine manually but fails with this error in a job? Any suggestions or best practices would be really helpful.

Thanks!

Akshay Petkar