- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-13-2026 12:00 PM
Hello,
Bronze: use classic or job compute, Autoloader with.option("mergeSchema", "true"). Schema evolution works correctly. data goes to bronze.my_bronze_table.
Silver: uses serverless compute, reader reads bronze.my_bronze_table, does all necessary transformation, writer creates silver.my_silver_table, which has defined schema.
The problem I am trying to resolve:
whenever the schema of bronze.my_bronze_table is changed (thanks to schema evolution), the writer fails to write silver.my_silver_table due to [STATE_STORE_VALUE_SCHEMA_NOT_COMPATIBLE] The provided value schema does not match existing schema in operator state.
The detailed debug log says clearly that the mismatch is in the schema of bronze.my_bronze_table.
As the silver uses serverless compute, I can not set