-werners-
Esteemed Contributor III

you can read the physical parquet files with spark.read.parquet().

The trick is to know which files are the current ones.