topic Re: Delta table version protocol in Data Engineering

Delta table version protocol

noname123 — Thu, 14 Mar 2024 12:37:30 GMT

I do:

df.write.format("delta").mode("append").partitionBy("timestamp").option("mergeSchema", "true").save(destination)

If table doesn't exist, it creates new table with "minReaderVersion":3,"minWriterVersion":7.

Yesterday it was creating table with "minReaderVersion":1,"minWriterVersion":2. I changed nothing and now versions are different. Setting delta protocol version on cluster does nothing.

This causes breaking change to me because some tables are used by Azure Data Factory and it's not compatible.

Re: Delta table version protocol

noname123 — Fri, 15 Mar 2024 12:22:28 GMT

Thanks for help.

Issue was caused by "Auto-Enable Deletion Vectors" setting.

Re: Delta table version protocol

AddBox45 — Thu, 10 Apr 2025 13:26:26 GMT

hello how did you fix this explicitly?how did you enable/disable the auto-enable deletion vectors setting to write again with minReaderVersion 1 and minWriterVersion 2?