โ10-19-2021 01:41 PM
Hi all,
I was reading the Repos documentation: https://docs.databricks.com/repos.html#migrate-from-run-commands
It is explained that, one advantage of Repos is no longer necessary to use %run magic command to make funcions available in one notebook to another. That is to say, we can import them with:
"from notebook_in_repos import fun"
I tested it out on Repos, but it doesnยดt work. I get: "No module named notebook_in_repos"
I really want this feature. It is painfull to build an entire module just to do an import. And on the other hand, the %run magic command is not a good replace; it overwrites local variables and mess up namespaces.
How can I make this work?
Thanks.
โ10-22-2021 07:25 AM
Thank you all for your help! I tried all that was suggested; but I finally realized it was my fault in first place:
Upgrading my runtime and migrating all my code to a .py file did the trick!
Thank you all again!
โ10-20-2021 01:38 AM
You can import the functions using the example mentioned in the blog. https://databricks.com/blog/2021/10/07/databricks-repos-is-now-generally-available.html
Let me know if this helps with your use case?
โ10-20-2021 02:59 AM
The thing in Repos is to point to the correct path.
Repos is just like in any local filesystem.
So in your case you need a notebook_in_repos folder with your .py files containing the functions.
โ10-20-2021 03:46 AM
My solution was to tell Python of that additional module import path by adding a snippet like this one to the notebook:
import os
import sys
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path:
sys.path.append(module_path)
This allows you to import the desired function from the module hierarchy:
from project1.lib.module import function
# use the function normally
function(...)
Note that it is necessary to add empty __init__.py files to project1/ and lib/ folders if you don't have them already.
If you want to know more about functions in Python go through EncodingCompiler
โ10-22-2021 07:25 AM
Thank you all for your help! I tried all that was suggested; but I finally realized it was my fault in first place:
Upgrading my runtime and migrating all my code to a .py file did the trick!
Thank you all again!
a week ago
Due to new functionalies in Runtime 16.0 regarding autoload i came across this autoload.
Performaed a practical test. It works. However had some problems at first.
As in solution the key was that definitions are places in a file.py not a notebook.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group