Anonymous
Not applicable
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-02-2021 04:34 PM
When and why should I convert b/w a Pandas to Koalas dataframe? What are the implications?
Labels:
- Labels:
-
Koalas Dataframe
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-04-2021 04:31 AM
Koalas is distributed on a Databricks cluster similar to how Spark dataframes are also distributed. Pandas dataframes only live on the spark driver in memory. If you are a pandas user and are using a multi-node cluster then you should use koalas to process the data. If you are able to use a single node databricks cluster then pandas could fit your needs as the data likely fits on a single computer.
