siva-anantha
Databricks Partner

@siva_pusarla: We use the following pattern and it works,

1) Calling notebook - constant location used by Job.

            + src/framework
                   + notebook_executor.py

2) Callee notebooks - dynamic

            + src/app/notebooks
                   + notebook1.py
                   + notebook2.py
                   ....
                   + notebookn.py

Job will send the notebook names as parameter to notebook_executor.py and it will execute notebooks using the below pattern,

notebook_path = f"../app/notebooks/{notebook_name}"
dbutils.notebook.run(notebook_path, TIMEOUT, input_params)

The important learning for us is the usage of relative path for notebook execution. Hope it helps