Anonymous
Not applicable
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-07-2023 10:21 PM
@Hanan Shteingart : To optimize queries with window functions, you should try multiple things like - filter the data, reduce the number of columns used in the window, optimize the window partitioning and ordering, and increase the cluster size if needed. Please try the below options
- FIlter the dataset df before applying the window function
- Reduce the number of columns in df before applying the window function
- Partition the dataframe df by choosing the right partition key that reduces the number of partitions and distributes the data evenly across the partitions
- Choose an ordering key that reduces the amount of data that needs to be sorted and processed
- Final resort, increase the size of your cluster to allocate more computing resources to the query