cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 
Help
Data Engineering
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 

What is the maximum size of a DataFrame that I can convert toPandas?

Gabriela_DeQuer
New Contributor

โ€Ž07-10-2015 02:19 PM

 
Labels:
0 Kudos
1 REPLY 1

User16826991422
Contributor

โ€Ž07-31-2015 10:21 AM

There is no hardcoded limit we just call panda.fromRecords with a collection of fields to instantiate a new Panda Dataframe.

The only limit is memory.

See http://stackoverflow.com/questions/15455722/pandas-is-there-a-max-size-max-no-of-columns-max-rows

0 Kudos
Announcements
Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.