- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-06-2025 06:41 AM - edited 08-06-2025 06:43 AM
I currently have a repo connected in databricks and I was able to correctly import a python module from src folder located in the same root.
Since I am using a Machine Learning runtime, I am force to choose a Dedicated (formerly: Single user) access mode. The module import works when I use directly my user, but if I change this into a group (to which I'm part of) then "src" module can't be find.
The weird thing is that when I run sys.path, I get the same exact list of paths.
So I am probably missing some knowledge of how the access works when using a ML runtime and assigning a group instead of a single user. Is there a way to import correctly the module?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-06-2025 07:32 AM - edited 08-06-2025 07:35 AM
Hi @FedeRaimondi ,
It could be permission issue. According to documentation, when a compute resource has Dedicated access, the resource can be assigned to a single user or a group. When assigned to a group (a group cluster), the user's permissions automatically down-scopes to the group's permissions, allowing the user to securely share the resource with other members of the group.
So maybe when you setup repo as a user the group has no access to that because of this down-scoped behaviour.
Databricks Runtime for Machine Learning | Databricks Documentation
I think this could be the case:
"For example, the group needs explicit permission to query a table, access a secret scope or secret, use a Unity Catalog connection credential, access a Git folder, or create a workspace object."
Assign compute resources to a group | Databricks Documentation
And you can check also best practices for managing group clusters:
Assign compute resources to a group | Databricks Documentation
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-06-2025 08:02 AM
Thanks @szymon_dybczak ! I confirm that's a permission issue and assigning "CAN MANAGE" solves it.
I still find it not really intuitive, since the goal is to use a shared cluster (with ML runtime) for development purposes. I mean, it would make sense when following the recommended approach of storing your code in "/Workspace/Groups/<groupName>", but for the integration with Git Folders it really adds some complexity.
The whole team assigned to the group will need to grant access to their Git Folders 😅
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-06-2025 08:14 AM
Hi @FedeRaimondi ,
Glad that it worked for you. And I agree with you that it's bit confusing 🙂
