Pyspark installation issue

[_r_vind1199]

When I try to start pyspark session in pycharm. It throws me this error "RuntimeError("Java gateway process exited before sending its port number"). Could anyone help me to solve this?

[Aashita · Accepted solution]

@Aravind A​ , What are the dependencies and versions you installed?

Install pyspark + openjdk

%pip install pyspark==2.4.8
!apt-get install openjdk-8-jdk-headless -qq > /dev/null

Build spark session

from pyspark.sql import SparkSession
spark = SparkSession.builder\
    .appName("Practice”)\
    .getOrCreate()

Note that:

Spark runs on Java 8/11, Scala 2.12/2.13, Python 3.6+ and R 3.5+. 

Python 3.6 support is deprecated as of Spark 3.2.0. 

Java 8 prior to version 8u201 support is deprecated as of Spark 3.2.0. 

For the Scala API, Spark 3.2.1 uses Scala 2.12. You will need to use a compatible Scala version (2.12.x). 

https://spark.apache.org/docs/latest/

[_r_vind1199]

@Aashita Ramteke​ , Pyspark version 3.2.1

[Aashita]

Kindly replace the version when you install python dependencies from 2.4.8 to 3.2.1.. It should work. Thanks.

[Kaniz]

Hi @Aravind A​ , Did you try all the steps as suggested by @Aashita Ramteke​ ?