cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Is there a function similar to display that downloads a dataframe?

jonathan-dufaul
Valued Contributor

I find myself constantly having to do display(df), and then "recompute with <5g records and download). I was just hoping I could skip the middleman and download from get go.

ideally it'd be a function like download(df,num_rows="max") where num_rows is either a number or the maximum (<5gb)

2 REPLIES 2

Hubert-Dudek
Esteemed Contributor III

Question where do you want to download it to? If to cloud location, use regular DataFrameWriter. You can install, for example, Azure Storage Explorer on your computer. Some cloud storage you can even mount in your system as a folder or network share.

oh no the download button on the dataframe itself to the local computer. I know how to write to every single location I need, but for quick data inspection > 1k rows it's a few clicks and a wasted query.

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now