Hey @Paul92S 

Looking at the error message it looks like column "metric_name" is the culprit here:

Understanding the Error:

• Character Limit Violation: The error indicates that values in the metric_name column are exceeding the maximum length allowed for char/varchar data types in Unity Catalog.
• Schema Mismatch: This likely stems from a discrepancy between the source data schema and the existing Delta table schema.

Troubleshooting Steps:

• Use source_df.printSchema() to verify the actual schema of your source data, paying close attention to the metric_name column's data type and length.
• Use spark.read.format("delta").load(destination_mount_filepath).printSchema() to inspect the current schema of the Delta table.
• If feasible, consider altering the metric_name column in the Delta table to accommodate longer values using ALTER TABLE TABLE_NAME ALTER COLUMN COLUMN_NAME;
• If increasing column size isn't viable, explore truncating values in the metric_name column before writing to the Delta table.
• If the error message refers to isnull('metric_name'), ensure there are no unintended null values in that column.
• If empty strings are valid for metric_name, explicitly set those values to empty strings in the source DataFrame.
• Also, check if the incremental files has a different schema vs the file you are using to create the table.

Follow-ups are appreciated!