cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Is there a function similar to display that downloads a dataframe?

jonathan-dufaul
Valued Contributor

I find myself constantly having to do display(df), and then "recompute with <5g records and download). I was just hoping I could skip the middleman and download from get go.

ideally it'd be a function like download(df,num_rows="max") where num_rows is either a number or the maximum (<5gb)

2 REPLIES 2

Hubert-Dudek
Esteemed Contributor III

Question where do you want to download it to? If to cloud location, use regular DataFrameWriter. You can install, for example, Azure Storage Explorer on your computer. Some cloud storage you can even mount in your system as a folder or network share.

oh no the download button on the dataframe itself to the local computer. I know how to write to every single location I need, but for quick data inspection > 1k rows it's a few clicks and a wasted query.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.