cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How to import a function to another notebook using Repos without %run?

maranBH
New Contributor III

Hi all,

I was reading the Repos documentation: https://docs.databricks.com/repos.html#migrate-from-run-commands

It is explained that, one advantage of Repos is no longer necessary to use %run magic command to make funcions available in one notebook to another. That is to say, we can import them with:

"from notebook_in_repos import fun"

I tested it out on Repos, but it doesnยดt work. I get: "No module named notebook_in_repos"

I really want this feature. It is painfull to build an entire module just to do an import. And on the other hand, the %run magic command is not a good replace; it overwrites local variables and mess up namespaces.

How can I make this work?

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

maranBH
New Contributor III

Thank you all for your help! I tried all that was suggested; but I finally realized it was my fault in first place:

  1. I was testing Files in Repos with a runtime < 8.4.
  2. I was trying to import a file from a DB Notebook instead of a static .py file.

Upgrading my runtime and migrating all my code to a .py file did the trick!

Thank you all again!

View solution in original post

4 REPLIES 4

User16829050420
Contributor

You can import the functions using the example mentioned in the blog. https://databricks.com/blog/2021/10/07/databricks-repos-is-now-generally-available.html

Let me know if this helps with your use case?

-werners-
Esteemed Contributor III

The thing in Repos is to point to the correct path.

Repos is just like in any local filesystem.

So in your case you need a notebook_in_repos folder with your .py files containing the functions.

madhuchennu
New Contributor III

My solution was to tell Python of that additional module import path by adding a snippet like this one to the notebook:

import os

import sys

module_path = os.path.abspath(os.path.join('..'))

if module_path not in sys.path:

  sys.path.append(module_path)

This allows you to import the desired function from the module hierarchy:

from project1.lib.module import function

# use the function normally

function(...)

Note that it is necessary to add empty __init__.py files to project1/ and lib/ folders if you don't have them already.

If you want to know more about functions in Python go through EncodingCompiler

maranBH
New Contributor III

Thank you all for your help! I tried all that was suggested; but I finally realized it was my fault in first place:

  1. I was testing Files in Repos with a runtime < 8.4.
  2. I was trying to import a file from a DB Notebook instead of a static .py file.

Upgrading my runtime and migrating all my code to a .py file did the trick!

Thank you all again!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group