cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Mount Workspace to Docker container

ivanychev
Contributor

Is there a way to mount Workspace folder (WSFS) to the Docker container if I'm using the Databricks Container Services ofr running a general purpose cluster?

If I create a cluster without a Docker image, the `!ls` command in Databricks notebook returns `/Workspace` folder that contains user folders, as well as their mounted Git repositories.

Is there a way to make this folder available when running a cluster with a Docker image? 

 

 

2 REPLIES 2

User16539034020
Contributor II
Contributor II

Hello:

Thanks for contacting Databricks Support!

I'm afraid that mounting the WSFS directly into a Docker container isn't directly supported. The Databricks workspace is a specialized environment and isn't directly analogous to a regular filesystem.

When you use custom containers in Databricks, your notebooks still run inside the Databricks workspace, and you can still access DBFS and other Databricks-specific features. The container is essentially providing the runtime environment, but you're still operating within the Databricks ecosystem. So, when you execute commands within a notebook, it's executed within that Databricks context, and you should still have access to the /Workspace and other Databricks-specific directories.

Regards,


So, when you execute commands within a notebook, it's executed within that Databricks context, and you should still have access to the /Workspace and other Databricks-specific directories.

This does not appear to be true. When I try to open any path in /Workspace from a python notebook, on a cluster running, for example, the image projectglow/databricks-glow:10.4, I get "No such file or directory" errors.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.