- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ10-19-2021 01:41 PM
Hi all,
I was reading the Repos documentation: https://docs.databricks.com/repos.html#migrate-from-run-commands
It is explained that, one advantage of Repos is no longer necessary to use %run magic command to make funcions available in one notebook to another. That is to say, we can import them with:
"from notebook_in_repos import fun"
I tested it out on Repos, but it doesnยดt work. I get: "No module named notebook_in_repos"
I really want this feature. It is painfull to build an entire module just to do an import. And on the other hand, the %run magic command is not a good replace; it overwrites local variables and mess up namespaces.
How can I make this work?
Thanks.
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ10-22-2021 07:25 AM
Thank you all for your help! I tried all that was suggested; but I finally realized it was my fault in first place:
- I was testing Files in Repos with a runtime < 8.4.
- I was trying to import a file from a DB Notebook instead of a static .py file.
Upgrading my runtime and migrating all my code to a .py file did the trick!
Thank you all again!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ10-20-2021 01:38 AM
You can import the functions using the example mentioned in the blog. https://databricks.com/blog/2021/10/07/databricks-repos-is-now-generally-available.html
Let me know if this helps with your use case?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ10-20-2021 02:59 AM
The thing in Repos is to point to the correct path.
Repos is just like in any local filesystem.
So in your case you need a notebook_in_repos folder with your .py files containing the functions.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ10-20-2021 03:46 AM
My solution was to tell Python of that additional module import path by adding a snippet like this one to the notebook:
import os
import sys
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path:
sys.path.append(module_path)
This allows you to import the desired function from the module hierarchy:
from project1.lib.module import function
# use the function normally
function(...)
Note that it is necessary to add empty __init__.py files to project1/ and lib/ folders if you don't have them already.
If you want to know more about functions in Python go through EncodingCompiler
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ10-22-2021 07:25 AM
Thank you all for your help! I tried all that was suggested; but I finally realized it was my fault in first place:
- I was testing Files in Repos with a runtime < 8.4.
- I was trying to import a file from a DB Notebook instead of a static .py file.
Upgrading my runtime and migrating all my code to a .py file did the trick!
Thank you all again!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ11-06-2024 02:25 AM
Due to new functionalies in Runtime 16.0 regarding autoload i came across this autoload.
Performaed a practical test. It works. However had some problems at first.
As in solution the key was that definitions are places in a file.py not a notebook.