Hubert-Dudek
Databricks MVP

official way is that before DROP:

  • Run DELETE FROM:

DELETE FROM events

  • Run VACUUM with an interval of zero:

VACUUM events RETAIN 0 HOURS

I agree that there could be some DEEP DROP 🙂

Alternatively not in SQL but in python you could write custom class/function to do that and then preinstall it on clusters so people would use some CleanTable(TableName) to make data validation and then delete+vacuum+drop+rm


My blog: https://databrickster.medium.com/

View solution in original post