Azure Databricks with Storage Account as data layer and DBFS understanding
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-07-2021 11:55 PM
- What is the difference between ADLS mounted ON DataBricks and dbfs
- does the Mount of ADLS on databricks make gives any performance benefit , is the mounted ADLS still behave as object storage or it become simple storage
Labels:
- Labels:
-
Azure databricks
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-07-2021 11:55 PM
- DBFS is just an abstraction on cloud storage
By default when you create a workspace, you get an instance of DBFS - so-called DBFS Root. Plus you can mount additional storage accounts under the
/mnt
folder. Data written to mount point paths (
/mnt
) is stored outside of the DBFS root. Even though the DBFS root is writeable, It's recommended that you store data in mounted object storage rather than in the DBFS root. The mounted storage does't change the performance of read and write , spark stills read the data in form of chunks and parallel processing is still applicable so there is no question of production degradation