Hi @SantiNath_Dey,

No... With cloudFiles.schemaEvolutionMode="addNewColumns", new columns are never put into the rescuedDataColumn. They always trigger an UnknownFieldException, Auto Loader writes an updated schema to schemaLocation, and then you restart the stream. There is no mode that both (a) evolves the schema and (b) rescues new columns instead of failing.

So you need a separate detection/logging step. A simple approach is to let Auto Loader run with addNewColumns (as you do now). After the schema has evolved and the table has been updated (after restart with mergeSchema), diff the table schema against a stored "previous schema snapshot" and log any new paths.

You can notify business users via email/alerts/dashboard, which can be built from the diff table created above. However, there’s no supported combination where addNewColumns will both evolve the schema and also send those new fields into the rescue column if that's what you are expecting.

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.