Hi Hubert,

The overwrite is not overwriting all folders, it only adds the new column to the re-written partitions.

The problem is that even I filter only the re-written partitions in the dataframe I do not see the new added columns. However, if I open one of the parquet files of the re-written partitions, I do see the new columns.

If I open one of the parquet files of the original partitions, I do not see the new columns.

I.e., the parquet files have the new column in the new partitions but not in the original partitions. That is something I would expect.

What I would expect and is not happening, is to get the new column when filtering only re-written partitions.

Thanks,

Tiago Rente.