@yuvesh kotiala​ :

The error message suggests that you need to use format("delta") instead of format("csv") when reading and writing to a Delta table. In your code, you are trying to write a Delta file as a CSV file, which is causing the error. If you want to write the data to a CSV file, you can first read the Delta file as a dataframe and then write it as a CSV file. Here's an example:

from pyspark.sql import SparkSession
 
# create a SparkSession
spark = SparkSession.builder.appName("DeltaToCSV").getOrCreate()
 
# read the Delta file as a dataframe
delta_df = spark.read.format("delta").load("s3://path/abc/")
 
# write the dataframe as a CSV file
delta_df.write.format("csv").mode("overwrite").options(delimiter="|").save(destinationBucketPath)

Note that when you read the Delta file, you need to use format("delta") and load() instead of csv() as in your original code. This will read the Delta file as a dataframe, which can then be written as a CSV file using format("csv") and save().