topic Requirement to Run Koalas in Data Engineering

Requirement to Run Koalas

User16826994223 — Wed, 23 Jun 2021 06:18:39 GMT

Hi I am planning to run Koalas on Databricks environment, What are the requirements for running Koalas there

Re: Requirement to Run Koalas

User16826994223 — Wed, 23 Jun 2021 06:19:04 GMT

Koalas is included on clusters running Databricks Runtime 7.3 and above.
To use Koalas on a cluster running Databricks Runtime 7.0 or below, install Koalas as an Azure Databricks PyPI library.
To use Koalas in an IDE, notebook server, or other custom applications that connect to an Azure Databricks cluster, install Databricks Connect and follow the Koalas installation instructions.

Re: Requirement to Run Koalas

tj-cycyota — Wed, 23 Jun 2021 15:05:03 GMT

Koalas is great! This really helps ease the transition from Pandas to Spark, because you can just use the same Pandas functions/classes through the Koalas API but everything runs in the background in Spark.