Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-27-2025 01:42 AM
Hi Mark,
Thanks for your response. I followed the steps you suggested:
- Created the table and set primary key + time series key constraints.
- Enabled Change Data Feed.
- Created the feature table and deployed the online endpoint — this worked fine.
- Removed some columns from the offline table and updated it using:
spark_df.write.mode("overwrite").option("overwriteSchema", "true").saveAsTable(f"{table_name}") - Updated the feature table using:
fe.write_table(name=feature_store_name, df=df, mode="merge") - Tried re-publishing to the online store using:
fe.publish_table(source_table_name=feature_store_list, online_table_name="catlog_dev.abcd.fs_table_online", online_store=pg_store, mode="merge")
— this step failed.
A LakeFlow pipeline was triggered and threw the following error:
org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] ... [DELTA_SCHEMA_CHANGED_WITH_STARTING_OPTIONS] Detected schema change in version 7
It seems the schema change isn’t being handled during re-publication. I’ve attached the full error message. Let me know if you need more details or logs.