Hi @dvd_lg_bricks ,There's a section about cluster configuration and tuning in Comprehensive Guide to Optimize Databricks, Spark and Delta Lake Workloads:Comprehensive Guide to Optimize Data Workloads | Databricks
Hi @Suheb ,Refer to really good guide prepared by Databricks team. When you have a skewed dataset the primary things you can do are following:1. Filter skewed values2. Apply Skew hints3. AQE skew optimization4. SaltingMuch detailed description of abo...
Hi @greengil ,You can't use functions to modify data. They're intended to return scalar value or table. If you need to modify content of a table use stored procedure instead.
