12-10-2021 03:30 PM
Hi @Tiago Rente
Have you try schema evolution? docs here https://spark.apache.org/docs/latest/sql-data-sources-parquet.html#schema-merging
I think that having your table as Delta will solve this issue. You might want to test it.
