https://community.databricks.com/t5/data-engineering/quot-hey-everyone-it-seems-like-there-39-s-some-confusion-about/m-p/8004#M3737 <P>"Hey everyone, it seems like there's some confusion about enhanced autoscaling in Databricks lately. If you're feeling lost or unsure about how it works, don't worry - you're not"</P><P>Enhanced autoscaling is a feature in Databricks that enables dynamic scaling of compute resources in response to changes in workload demand. Unlike traditional autoscaling, which simply adds or removes instances based on pre-set thresholds, enhanced autoscaling uses machine learning algorithms to make more accurate predictions about future workload demand and adjust the number of instances accordingly.</P><P>This allows for more efficient resource utilization and cost savings, as well as improved performance and faster job completion times. Enhanced autoscaling also supports custom policies, allowing users to tailor the scaling behavior to their specific use case and requirements. Overall, it is a powerful tool for optimizing the use of cloud resources and maximizing the value of Databricks for data processing and analyse</P><P></P><P>do follow for such more posts Rishabh Pandey </P><P>​</P><P><B>#machinelearning</B> <B>#databricks</B> <B>#bigdata</B> @Sujitha Ramamoorthy​&nbsp;@Kaniz Fatma​&nbsp;@Ajay Pandey​&nbsp;@Aviral Bhardwaj​&nbsp;</P> Thu, 09 Mar 2023 16:19:11 GMT Rishabh-Pandey 2023-03-09T16:19:11Z https://community.databricks.com/t5/data-engineering/quot-hey-everyone-it-seems-like-there-39-s-some-confusion-about/m-p/8004#M3737 <P>"Hey everyone, it seems like there's some confusion about enhanced autoscaling in Databricks lately. If you're feeling lost or unsure about how it works, don't worry - you're not"</P><P>Enhanced autoscaling is a feature in Databricks that enables dynamic scaling of compute resources in response to changes in workload demand. Unlike traditional autoscaling, which simply adds or removes instances based on pre-set thresholds, enhanced autoscaling uses machine learning algorithms to make more accurate predictions about future workload demand and adjust the number of instances accordingly.</P><P>This allows for more efficient resource utilization and cost savings, as well as improved performance and faster job completion times. Enhanced autoscaling also supports custom policies, allowing users to tailor the scaling behavior to their specific use case and requirements. Overall, it is a powerful tool for optimizing the use of cloud resources and maximizing the value of Databricks for data processing and analyse</P><P></P><P>do follow for such more posts Rishabh Pandey </P><P>​</P><P><B>#machinelearning</B> <B>#databricks</B> <B>#bigdata</B> @Sujitha Ramamoorthy​&nbsp;@Kaniz Fatma​&nbsp;@Ajay Pandey​&nbsp;@Aviral Bhardwaj​&nbsp;</P> Thu, 09 Mar 2023 16:19:11 GMT https://community.databricks.com/t5/data-engineering/quot-hey-everyone-it-seems-like-there-39-s-some-confusion-about/m-p/8004#M3737 Rishabh-Pandey 2023-03-09T16:19:11Z https://community.databricks.com/t5/data-engineering/quot-hey-everyone-it-seems-like-there-39-s-some-confusion-about/m-p/8005#M3738 <P>Very informative</P><P>Thanks for sharing!</P> Fri, 10 Mar 2023 04:07:41 GMT https://community.databricks.com/t5/data-engineering/quot-hey-everyone-it-seems-like-there-39-s-some-confusion-about/m-p/8005#M3738 Ajay-Pandey 2023-03-10T04:07:41Z https://community.databricks.com/t5/data-engineering/quot-hey-everyone-it-seems-like-there-39-s-some-confusion-about/m-p/8006#M3739 <P>Thanks @Ajay Pandey​&nbsp;​</P> Sat, 11 Mar 2023 08:15:16 GMT https://community.databricks.com/t5/data-engineering/quot-hey-everyone-it-seems-like-there-39-s-some-confusion-about/m-p/8006#M3739 Rishabh-Pandey 2023-03-11T08:15:16Z