Hi @NarenderKumar ,

Have you considered leveraging autoscaling for the existing cluster?

If this does not meet your needs, are the differing volume/workloads known in advance? If so, could different compute be provisioned using Infrastructure as Code based on the differing workload characteristics? Here's a doc on using Terraform with Databricks: https://docs.databricks.com/en/dev-tools/terraform/index.html

Thank you.