cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

What is DBFS? What is the purpose of it and what does it allow me to do?

Anonymous
Not applicable
 
1 ACCEPTED SOLUTION

Accepted Solutions

sean_owen
Honored Contributor II
Honored Contributor II

DBFS is the "Databricks File System", but really it's just a shim / wrapper on top of distributed storage, that makes files in S3 or ADLS look like local files under the path /dbfs/... This can be really useful when working with libraries that do not understand distributed storage. They can read/write distributed storage as if it's a local file. DBFS also lets you mount other buckets under different paths under /dbfs. That's about all it is though, just a translation layer that exposes distributed storage under /dbfs (or under dbfs:/ URIs)

View solution in original post

1 REPLY 1

sean_owen
Honored Contributor II
Honored Contributor II

DBFS is the "Databricks File System", but really it's just a shim / wrapper on top of distributed storage, that makes files in S3 or ADLS look like local files under the path /dbfs/... This can be really useful when working with libraries that do not understand distributed storage. They can read/write distributed storage as if it's a local file. DBFS also lets you mount other buckets under different paths under /dbfs. That's about all it is though, just a translation layer that exposes distributed storage under /dbfs (or under dbfs:/ URIs)

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.