Hubert-Dudek
Databricks MVP

‎10-20-2022 05:30 AM

  • If you append data to the partition folder, which is the date, then it makes sense to Optimize only that folder (so WHERE is equal to appended date), but if it is not partitioned, I will skip WHERE option,
  • VACUUM cleans the history of the table. So you can adjust the retention period. In some cases, like Bronze data, I VACUUM daily with RETAIN 0, but essential data, especially data under some regulations, can never be VACUUM. So every case can be different.

My blog: https://databrickster.medium.com/