Hi,

I would highly recommend to check this doc https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/index.html you can use Pandas APIs on Spark. It will have a huge benefit when you compare with standard Pandas, which only runs on the driver. In this docs, you will be able to find some examples on how to read data using Pandas.