cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

What is DBFS? What is the purpose of it and what does it allow me to do?

Anonymous
Not applicable
 
1 ACCEPTED SOLUTION

Accepted Solutions

sean_owen
Databricks Employee
Databricks Employee

DBFS is the "Databricks File System", but really it's just a shim / wrapper on top of distributed storage, that makes files in S3 or ADLS look like local files under the path /dbfs/... This can be really useful when working with libraries that do not understand distributed storage. They can read/write distributed storage as if it's a local file. DBFS also lets you mount other buckets under different paths under /dbfs. That's about all it is though, just a translation layer that exposes distributed storage under /dbfs (or under dbfs:/ URIs)

View solution in original post

1 REPLY 1

sean_owen
Databricks Employee
Databricks Employee

DBFS is the "Databricks File System", but really it's just a shim / wrapper on top of distributed storage, that makes files in S3 or ADLS look like local files under the path /dbfs/... This can be really useful when working with libraries that do not understand distributed storage. They can read/write distributed storage as if it's a local file. DBFS also lets you mount other buckets under different paths under /dbfs. That's about all it is though, just a translation layer that exposes distributed storage under /dbfs (or under dbfs:/ URIs)

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group