Hi @NhanNguyen .  If you already have some pandas code written to do the data quality check, then I'd say that you'd lean toward method 2.  But if you are starting from scratch and comfortable coding in PySpark (without Pandas) then I'd say you'd go with method 4.  When you say "check of each file", you'd need to have some group-by variable that designated each file.  Does this answer your question?