Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-25-2021 03:05 AM
Typically, Spark (Databricks) is not made for downloading files locally to your laptop.
It is a distributed computing system optimized for parallel writes to some kind of storage (DBFS for Databricks).
I do not know what your use case is exactly, but if you want to download data, it might be a good idea to let Databricks write to a data lake/blob storage etc (something mounted in DBFS).
From there on, you can download it to your computer if necessary.
There is the possibility to download data from within the notebooks (display command), but I think there is a hard limit in the amount of data which can be transferred that way.