This is a really interesting breakdown of the S3 request-cost problem. I’d also be cautious about simply reducing the commit frequency, since that can solve the cost issue while creating a freshness problem.I’d check the exact Databricks Runtime vers...
