Service Principal cannot access its own workspace folder
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-29-2025 02:07 PM
We are using Asset bundles with databricks runtime 14.3LTS. During DAB deployment, the wheel is built and stored in the folder of the service principal running the deployment via GH workflow. The full path is
/Workspace/Users/SERVICE-PRINCIPAL-ID/.bundle/main_bundle/dev_dev/artifacts/.internal/WHEEL-NAME.whl
Within the same DAB, we define and deploy a job which uses a job cluster. when using the wheel as a task dependency we get this error message:
Spoiler
run failed with error message Library installation failed for library due to user error for whl: "/Workspace/Users/SERVICE-PRINCIPAL-ID/.bundle/main_bundle/dev_dev/artifacts/.internal/WHEEL-NAME.whl" Error messages: Library installation attempted on the driver node of cluster CLUSTER-ID and failed. User does not have permission to read the library file, or the file path does not exist. Error Code: FILE_NOT_FOUND_FAILURE. Error Message: java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File file:/Workspace/Users/SERVICE-PRINCIPAL-ID/.bundle/main_bundle/dev_dev/artifacts/.internalWHEEL-NAMEwhl does not exist
This was working fine a couple days before and now it is failing out of nowhere.
