We are migrating our data into Databricks and I was looking at the recommendations for partitioning here: https://docs.databricks.com/tables/partitions.html. This recommends not specifying partitioning and allowing "Ingestion Time Partitioning" (ITP) to optimize data files. However, details on this are slim. It seems like the data should be optimized by some timestamp but at the time of initial load the data all has the same ingest time. The data sources will mainly be change data capture data which will have an `updated_on` or other transaction timestamp that queries will use to get the latest copy of each record. I'd like to take advantage of ITP, but on the initial load it seems like I should indicate a specific column that should be used - like when I would call optimize. Should I just call optimize after my initial load? If I do will that disable ITP for the table? If not, what is the recommendation?
