Delta Lake Python

[delta_lake]

I have setup a virtual environment inside my existing hadoop cluster. Since the current cluster does not have spark >3 , so i installed delta spark using virtual environment. While trying to access the hdfs which is kerberose one, Getting below error. Can you one help to advise on this

Client auth mode SIMPLE, mechanism is not supported on zone, Auth:kerberos_only

from delta import *
    
builder = pyspark.sql.SparkSession.builder.appName("MyApp") \
   .config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension") \
   .config("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.delta.catalog.DeltaCatalog")
    
spark = configure_spark_with_delta_pip(builder).getOrCreate()
spark.read.parquet("hdfs/parquet_path")

[Kaniz]

Hi @ delta lake! My name is Kaniz, and I'm the technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers on the community have an answer to your question first. Or else I will follow up with my team and get back to you soon.Thanks.

[User16753724663]

Hi @Vasanth P​ ,

Could you please confirm the DBR version that you are using in the cluster?

There are multiple DBR versions that is available with different spark and scala version which can utilised directly in databricks cluster.

[Kaniz]

Hi @Vasanth P​ ​ , Just a friendly follow-up. Do you still need help or the above responses help you to find the solution? Please let us know.