cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Is there a function similar to display that downloads a dataframe?

jonathan-dufaul
Valued Contributor

I find myself constantly having to do display(df), and then "recompute with <5g records and download). I was just hoping I could skip the middleman and download from get go.

ideally it'd be a function like download(df,num_rows="max") where num_rows is either a number or the maximum (<5gb)

2 REPLIES 2

Hubert-Dudek
Esteemed Contributor III

Question where do you want to download it to? If to cloud location, use regular DataFrameWriter. You can install, for example, Azure Storage Explorer on your computer. Some cloud storage you can even mount in your system as a folder or network share.

oh no the download button on the dataframe itself to the local computer. I know how to write to every single location I need, but for quick data inspection > 1k rows it's a few clicks and a wasted query.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group