Is there a function similar to display that downloads a dataframe?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-24-2023 07:28 AM
I find myself constantly having to do display(df), and then "recompute with <5g records and download). I was just hoping I could skip the middleman and download from get go.
ideally it'd be a function like download(df,num_rows="max") where num_rows is either a number or the maximum (<5gb)
- Labels:
-
Dataframe
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-24-2023 10:26 AM
Question where do you want to download it to? If to cloud location, use regular DataFrameWriter. You can install, for example, Azure Storage Explorer on your computer. Some cloud storage you can even mount in your system as a folder or network share.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-24-2023 11:15 AM
oh no the download button on the dataframe itself to the local computer. I know how to write to every single location I need, but for quick data inspection > 1k rows it's a few clicks and a wasted query.

