04-14-2022 11:25 AM
When I try to start pyspark session in pycharm. It throws me this error "RuntimeError("Java gateway process exited before sending its port number"). Could anyone help me to solve this?
04-14-2022 04:31 PM
@Aravind A , What are the dependencies and versions you installed?
Install pyspark + openjdk
%pip install pyspark==2.4.8
!apt-get install openjdk-8-jdk-headless -qq > /dev/null
Build spark session
from pyspark.sql import SparkSession
spark = SparkSession.builder\
.appName("Practice”)\
.getOrCreate()
Note that:
Spark runs on Java 8/11, Scala 2.12/2.13, Python 3.6+ and R 3.5+.
Python 3.6 support is deprecated as of Spark 3.2.0.
Java 8 prior to version 8u201 support is deprecated as of Spark 3.2.0.
For the Scala API, Spark 3.2.1 uses Scala 2.12. You will need to use a compatible Scala version (2.12.x).
04-14-2022 04:31 PM
@Aravind A , What are the dependencies and versions you installed?
Install pyspark + openjdk
%pip install pyspark==2.4.8
!apt-get install openjdk-8-jdk-headless -qq > /dev/null
Build spark session
from pyspark.sql import SparkSession
spark = SparkSession.builder\
.appName("Practice”)\
.getOrCreate()
Note that:
Spark runs on Java 8/11, Scala 2.12/2.13, Python 3.6+ and R 3.5+.
Python 3.6 support is deprecated as of Spark 3.2.0.
Java 8 prior to version 8u201 support is deprecated as of Spark 3.2.0.
For the Scala API, Spark 3.2.1 uses Scala 2.12. You will need to use a compatible Scala version (2.12.x).
04-15-2022 06:47 AM
@Aashita Ramteke , Pyspark version 3.2.1
04-20-2022 09:36 AM
Kindly replace the version when you install python dependencies from 2.4.8 to 3.2.1.. It should work. Thanks.
04-18-2022 11:26 PM
Hi @Aravind A , Did you try all the steps as suggested by @Aashita Ramteke ?
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group