What is the maximum size of a DataFrame that I can convert toPandas?

Gabriela_DeQuer
New Contributor
 

rlgarris
Databricks Employee
Databricks Employee

There is no hardcoded limit we just call panda.fromRecords with a collection of fields to instantiate a new Panda Dataframe.

The only limit is memory.

See http://stackoverflow.com/questions/15455722/pandas-is-there-a-max-size-max-no-of-columns-max-rows