Hi @-werners- ,
In case the problem is actually caused by these filters because DLT applies these configs for data quality, is there a way to ignore these filters when reading data and ignore delta_log files when reading in that partition?
I want to verify to make sure that DLT automatically adding these filters will cause performance problems in processing data with Spark.