@Shelly Bhardwaj​ :

The error IndexError: tuple index out of range suggests that there is an issue with the serialization of the RDD using PySpark's cloudpickle library. This can happen when the size of the data being serialized exceeds the maximum size limit for cloudpickle. One way to overcome this issue is to use a different serialization method. You can try using PySpark's pickle serializer instead of cloudpickle. You can set the serializer using the SparkConf object before creating the SparkContext:

from pyspark import SparkConf, SparkContext
 
conf = SparkConf().setAppName("myApp").set("spark.serializer", "org.apache.spark.serializer.PickleSerializer")
sc = SparkContext(conf=conf)
 
x = [1, 2, 3, 4, 5, 6, 7]
rdd = sc.parallelize(x)
print(rdd.take(2))

Alternatively, you can try reducing the size of the RDD by filtering or partitioning the data before serializing it.