Thanks for reply. Can you suggest consumers when reading custom code to read files can get partitional column?

presently consumer is getting list of all files in folder, and filtering out files already processed and then read each new file with

spark.read.format('parquet').load(filePath)