cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

UC volumes not useable in Apps?

nk-five1
Visitor

I have to install some custom library in a Python Databricks App. According to the documentation this should be possible through UC volumes:
https://docs.databricks.com/aws/en/dev-tools/databricks-apps/dependencies#install-wheel-files-from-u...

However, the volume is never mounted inside the app:
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/Volumes/catalog/schema/volume/package_name-1.0.0-py3-none-any.whl'
Listing the root directory in the app (ls /) and showing the result with streamlit confirms that there is no directory /Volume in the app.

Is there any other way to install an app dependency without having to copy the source code or wheel into the app code?
Does anyone have any idea how to get my package installed in the app? Setting up a private repository to point pip to is not an option in this scenario.

Thanks, Nico

4 REPLIES 4

nk-five1
Visitor

I have confirmed that the app's service principal has READ access to the volume, the whl is readable through the SDK and the app has a ressource UC volume that is referenced in app.yml and available as an env var within the app.
According to the documentation that should be all prerequisites and the installation should work:
https://docs.databricks.com/aws/en/dev-tools/databricks-apps/dependencies#install-wheel-files-from-u...

szymon_dybczak
Esteemed Contributor III

Hi @nk-five1 ,

This is a known issue. It seems that UC volumes are not mounted and you cannot access them using python os module. You can read more at below thread:

https://community.databricks.com/t5/data-governance/accessing-permanent-file-system-from-databricks-...

The workaround for now is to use python SDK

nk-five1
Visitor

Thank you. So I guess the workaround is to pull the .whl into the app using the SDK, install it manually and only then start the actual entrypoint. Very tedious and error prone. That even makes duplicating the dependency code into the app seem like a not-so-bad alternative ๐Ÿ˜„

szymon_dybczak
Esteemed Contributor III

I hope that in near future they add possibility to access volumes using plain python os module. Because as of now interacting with volumes from databricks apps is quite cumbersome