How to import one databricks python notebook into another?

[amanpreetkaur]

I have a python notebook A in Azure Databricks having import statement as below:

import xyz, datetime,...

I have another notebook xyz being imported in notebook A as shown in above code. When I run notebook A, it throws the following error:

ImportError:No module named xyz  

Both notebooks are in the same workspace directory. Can anyone help in resolving this?

[amanpreetkaur]

Can you please answer this? @jurmu

[kinka007]

savastan0 cc shop

[Julio_Batista]

I belive this is not possible. You can run a notebook passing parameters as:

result = dbutils.notebook.run("notebook-name", 60, {"argument": "data", "argument2": "data2", ...})

But notebooks can only return strings. What you need is to upload a Python module as a library and them import the module in your notebook.

[MikhailKolomaso]

For me worked well solution:

1) Create library notebook. For example - "Lib" with any functions/classes there (no runnable code).

2) Create main notebook. For example - "Main"

3) To import into main all classes & functions from Lib to Main use command:

%run "./Lib"

(this will works like: from Lib import *)

4) After that you can call any functions/ use classes that used in Lib from Main notebook.

PS:

1.1) recursion enabled - i.e. you lib notebook may contain code that runs any other notebooks the same way

1.2) for reload changed code from Lib module - just re-run command %run "./Lib"

[awaiskaleem]

@Mikhail Kolomasov​ , I tried exactly this, it worked until step 4. I'm trying to import variables in a py file into my "Main" notebook. What am i missing here?

2019-12-30-12-18-45-emr-pseg.png

[virgilus]

Thank you it worked like a charm! 🙂

[virgilus]

Thanks, worked like a charm! 🙂

[m96]

Created a package to do exactly this. Have a look: https://pypi.org/project/libify/

(Relies on dbutils.notebook.run, so won't work with the Databricks community edition)

[ramravi]

NoebookA:

def func1():
 
     pass

NotebookB:

%run /NotebookA
 
func1()

[artsheiko]

1. Create a repository containing an __init__.py file
2. Add your library as .py file(s). Let's imagine that our library is composed by multiple sub-folders consolidated in "my_folder", one of sub-folders is named as "math_library" and contains my_awesome_lib.py
3. In caller Notebook import as a module :

import os
import sys
sys.path.append(os.path.abspath("/Workspace/Repos/<usename>/<repo-name>/<path>/my_folder"))

or use pathlib to adress the folder you need

import os
import sys
from pathlib import Path
 
sys.path.append(str(Path(os.getcwd()).parent.absolute()))

4. Make a call to the function in question :

from math_library import my_awesome_lib
 
my_awesome_lib.<function needed name>()

[Mumrel]

Why does the description here look much simpler than described above?

[artsheiko]

The documentation article covers the case when the file to import is located in the same directory. The step-by-step process I described shows how to import a file from an independent repository. Both are same in principle

[Mumrel]

Got it, thanks for the quick response

[Tim3]

I followed those descriptions and got the error "[Errno 95] Operation not supported" on the import line using DBR 12.2 LTS. I think there may be a bug in the runtime or the content of the descriptions is obsolete.