Databricks is highly optimized for Delta, which leverages columnar storage, indexing, and caching for better performance.Instead of directly processing CSV files, convert them to Delta first, then perform aggregations and joins, see if this helps