My source is : Oracle db - Golden gate - kafka - Flink - S3 - bronze - silver - gold
It's a live cdc data, in S3 we have delta tables in string format, bronze also having string format for all tables, it's a DLT pipeline
Now we have to map datatypes and table structure like existing columns and new column arrival in silver dynamically continuously without delay of Microsoft seconds, pipelines and jobs we can not stop.
Note - we have one metadata snapshot table in S3, which we can consume to map in silver
Kindly advise how to achieve this