Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-01-2022 05:33 AM
I agree with the @Retko Okter
To support the second point, find the below explanation,
Optimized autoscaling
- Firstly, scales up from min to max in 2 steps.
- Secondly, can scale down even if the cluster is not idle by looking at shuffle file state.
- Thirdly, scales down based on a percentage of current nodes.
- And, on job clusters, scales down if the cluster is underutilized over the last 40 seconds.
- Lastly, on all-purpose clusters, scales down if the cluster is underutilized over the last 150 seconds.
Standard autoscaling
- Firstly, starts with adding 8 nodes. Thereafter, scales up exponentially, but can take many steps to reach the max.
- Secondly, scales down only when the cluster is completely idle and it has been underutilized for the last 10 minutes.
- Lastly, scales down exponentially, starting with 1 node.