How to import a function to another notebook using Repos without %run?

maranBH
New Contributor III

‎10-19-2021 01:41 PM

Hi all,

I was reading the Repos documentation: https://docs.databricks.com/repos.html#migrate-from-run-commands

It is explained that, one advantage of Repos is no longer necessary to use %run magic command to make funcions available in one notebook to another. That is to say, we can import them with:

"from notebook_in_repos import fun"

I tested it out on Repos, but it doesn´t work. I get: "No module named notebook_in_repos"

I really want this feature. It is painfull to build an entire module just to do an import. And on the other hand, the %run magic command is not a good replace; it overwrites local variables and mess up namespaces.

How can I make this work?

Thanks.

Labels:
0 Kudos

User16829050420
Databricks Employee
Databricks Employee

‎10-20-2021 01:38 AM

You can import the functions using the example mentioned in the blog. https://databricks.com/blog/2021/10/07/databricks-repos-is-now-generally-available.html

Let me know if this helps with your use case?

-werners-
Esteemed Contributor III

‎10-20-2021 02:59 AM

The thing in Repos is to point to the correct path.

Repos is just like in any local filesystem.

So in your case you need a notebook_in_repos folder with your .py files containing the functions.

madhuchennu
New Contributor III

‎10-20-2021 03:46 AM

My solution was to tell Python of that additional module import path by adding a snippet like this one to the notebook:

import os

import sys

module_path = os.path.abspath(os.path.join('..'))

if module_path not in sys.path:

  sys.path.append(module_path)

This allows you to import the desired function from the module hierarchy:

from project1.lib.module import function

# use the function normally

function(...)

Note that it is necessary to add empty __init__.py files to project1/ and lib/ folders if you don't have them already.

If you want to know more about functions in Python go through EncodingCompiler

maranBH
New Contributor III

‎10-22-2021 07:25 AM

Thank you all for your help! I tried all that was suggested; but I finally realized it was my fault in first place:

  1. I was testing Files in Repos with a runtime < 8.4.
  2. I was trying to import a file from a DB Notebook instead of a static .py file.

Upgrading my runtime and migrating all my code to a .py file did the trick!

Thank you all again!

View solution in original post

JakubSkibicki
Contributor

‎11-06-2024 02:25 AM

Due to new functionalies in Runtime 16.0 regarding autoload i came across this autoload. 

Performaed a practical test. It works. However had some problems at first.

As in solution the key was that definitions are places in a file.py not a notebook.

0 Kudos