TypeError: 'JavaPackage' object is not callable

[farazahmad372]

from pyspark.sql import *

if __name__ == "__main__":

    spark = SparkSession.builder \
        .appName("hello Spark") \
        .master("local[2]") \
        .getOrCreate()

    data_list = [("Ravi",28),
                 ("David",45),
                 ("Abdul",37),
                ]

    df = spark.createDataFrame(data_list).toDF("Name","age")

    df.show()

error --->

D:\SparkPractice\venv\Scripts\python.exe D:\SparkPractice\practice.py
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Traceback (most recent call last):
File "D:\SparkPractice\practice.py", line 3, in <module>
spark = SparkSession.builder.master("local[1]").appName("Test").getOrCreate()
File "D:\SparkPractice\venv\Lib\site-packages\pyspark\sql\session.py", line 559, in getOrCreate
session = SparkSession(sc, options=self._options)
File "D:\SparkPractice\venv\Lib\site-packages\pyspark\sql\session.py", line 635, in __init__
jSparkSessionClass.getDefaultSession().isDefined()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
TypeError: 'JavaPackage' object is not callable

Process finished with exit code 1

[nikhilj0421]

Hi @farazahmad372, which Databricks runtime version are you using? I just tried it out in 14.3, no isolation mode, and it works as expected. 

[farazahmad372]

Hi @nikhilj0421 , I am facing the below issue now , on the above code

raceback (most recent call last):
File "D:\SparkPractice\HelloSpark.py", line 4, in <module>
from pyspark.sql import SparkSession
ImportError: cannot import name 'SparkSession' from 'pyspark.sql' (unknown location)

[nikhilj0421]

@farazahmad372 May I know the DBR version and type of cluster?
Are you using serverless?