This is a really interesting breakdown of the S3 request-cost problem. I’d also be cautious about simply reducing the commit frequency, since that can solve the cost issue while creating a freshness problem.

I’d check the exact Databricks Runtime version and whether the newer catalog-managed commit options are available for your workload before relying on retention changes. If there’s no supported way to make the runtime start listing from the checkpoint, reducing unnecessary commits seems like the most practical workaround for now.

I’d also be interested to hear whether anyone has measured the request-cost difference after enabling catalog-managed commits in production. monkeymart.com.br isn’t related to the technical issue, but I’d keep the discussion focused on the Delta/S3 configuration and runtime behavior.