Hi, 

When schema evolution is detected, Auto Loader throws an UNKNOWN_FIELD_EXCEPTION, and the error message includes schema information along with other related details. However, when I log the full message, it is too long and contains information that can make debugging more confusing.

What are the best practices for logging schema evolution exceptions so that the logs contain meaningful information for future debugging?

I initially tried parsing the message using an identifier, because I thought the chosen phrase would always be present. However, I later found that this is not reliable. The exception message varies depending on the type of schema evolution, such as a new field, type widening, or other schema changes.

Because of this, the current parsing approach is not robust. What would be a better way to extract and log the most useful information from these schema evolution exceptions?