Brahmareddy
Esteemed Contributor II

Hi @RoelofvS,

How are you doing today? As per my understanding, You just make sure cloudFiles.schemaEvolutionMode is set to addNewColumns to enable automatic schema updates for new columns. If schema versions aren't updating in the same location, try pointing to a new schema location to reset schema tracking temporarily, though this shouldn’t be needed under normal conditions. Check file access permissions in the inferred_schema directory as permissions issues could prevent schema updates. Running the demo on a fresh cluster or with a different schema location can help identify if the environment is impacting schema evolution. Lastly, consider testing with a different Databricks runtime version to rule out runtime-specific issues.

Give a try and let me know.

Regards,

Brahma