Hey,
Okay thanks @nikhilj0421. I have now solved the issue but not with a full refresh of the table. I had tried this previously and even deleted the DLT pipeline hoping that would provide me the clean slate if this lingering schema was an issue but when re-running the pipeline it still failed with the same issue. The solution was to change the output schema(in my case to {schema_name}_2) of the table/pipeline and run again. I was then able to again alter the schema back to the original schema_name and this time everything worked without a problem.

Overall I have learned that DLT pipelines must store more information than I previously thought when deleting them or full refreshing them.