How to disable spark connect in the databricks compute?

[chinmay0924]

I want to be able to access the RDD methods of a Dataframe, but it seems that this is not supported in spark connect. I have been trying to disable spark connect in the spark config using,

spark.databricks.service.server.enabled false

but when I check the type of spark it is still `pyspark.sql.connect.session.SparkSession`. How do I disable spark connect?

[Bareaj]

If you only want to disable it, set spark.databricks.service.server.enabled to false. However, you might not be able to run your code afterward

[Bareaj]

Maybe you can do something like:

from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()

if spark.__class__.__module__ == "pyspark.sql.connect.session":

    code when it is enable

else:

    code when not

[chinmay0924]

I want to disable and I tried setting spark.databricks.service.server.enabled to false in the spark config while creating the compute, but when I run the notebook with this compute and I do 

type(spark), I get pyspark.sql.connect.session.SparkSession.

[Bareaj]

I have found that when the cluster is shared, it automatically uses that type of session, and in that case, I have not been able to disable it. I don't know if this is your situation. I have avoided some problems that I had with the previous clause.