Mount Workspace to Docker container

ivanychev
Contributor II

‎08-23-2023 05:58 PM

Is there a way to mount Workspace folder (WSFS) to the Docker container if I'm using the Databricks Container Services ofr running a general purpose cluster?

If I create a cluster without a Docker image, the `!ls` command in Databricks notebook returns `/Workspace` folder that contains user folders, as well as their mounted Git repositories.

Is there a way to make this folder available when running a cluster with a Docker image? 

 

 

Sergey
0 Kudos

User16539034020
Databricks Employee
Databricks Employee

‎11-02-2023 04:12 PM

Hello:

Thanks for contacting Databricks Support!

I'm afraid that mounting the WSFS directly into a Docker container isn't directly supported. The Databricks workspace is a specialized environment and isn't directly analogous to a regular filesystem.

When you use custom containers in Databricks, your notebooks still run inside the Databricks workspace, and you can still access DBFS and other Databricks-specific features. The container is essentially providing the runtime environment, but you're still operating within the Databricks ecosystem. So, when you execute commands within a notebook, it's executed within that Databricks context, and you should still have access to the /Workspace and other Databricks-specific directories.

Regards,

0 Kudos

IgorP-NVTA
New Contributor II
In response to User16539034020

‎11-22-2023 09:19 AM - edited ‎11-22-2023 09:21 AM

So, when you execute commands within a notebook, it's executed within that Databricks context, and you should still have access to the /Workspace and other Databricks-specific directories.

This does not appear to be true. When I try to open any path in /Workspace from a python notebook, on a cluster running, for example, the image projectglow/databricks-glow:10.4, I get "No such file or directory" errors.

0 Kudos