Auto-scaling not getting kicked in

- - Certifications
- - Learning Paths
- - Databricks Product Tours
- - Get Started Guides
- - Product Platform Updates

- - Get Started Resources
- - Events
- - Support FAQs
- - Technical Blog
- - Knowledge Sharing Hub
- - Announcements
- - DatabricksTV

- - Private Groups
  - Farfetch Databricks User Community
- - Skills@Scale

- - Databricks Community Champions
- - Khoros Community Forums Support (Not for Databricks Product Questions)
- - Databricks Community Code of Conduct

Data Engineering

Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.

I have a spark-submit job, I do not see auto-scaling happening on the cluster at the time of executions.

1 ACCEPTED SOLUTION

Accepted Solutions

This is working as expected. Autoscaling is not available for spark-submit jobs

Run the job as jar job instead of spark-submit

View solution in original post

1 REPLY 1

This is working as expected. Autoscaling is not available for spark-submit jobs

Run the job as jar job instead of spark-submit

never-displayed

You must be signed in to add attachments

never-displayed

Announcements

Announcing the APJ Databricks Smart Business Insights Challenge: Empowering Data-Driven Decision Mak

🚀 Monthly Databricks Get Started Days – Accelerate Your Learning Journey! 🚀

Virtual Learning Festival: 9 April - 30 April

Data + AI Summit 2025 — registration now open!

Databricks Community

Auto-scaling not getting kicked in

Join Us as a Local Community Builder!

Announcing the APJ Databricks Smart Business Insights Challenge: Empowering Data-Driven Decision Mak

🚀 Monthly Databricks Get Started Days – Accelerate Your Learning Journey! 🚀

Business Intelligence in the Era of AI

Virtual Learning Festival: 9 April - 30 April

Data + AI Summit 2025 — registration now open!