Hi @Mohan2, Based on the errors you're encountering, you're having issues with cluster creation and quota limitations.
Here are some potential solutions:
1. **Increase your Azure quota:** The error message indicates that your Azure subscription does not have enough percentage to create the needed resources. You can request a quota increase by contacting Azure Support.
2. **Setup Virtual Machines in Azure:** Since you mentioned you don't have Virtual VMs set up in Azure, you might need to set them up.
Here are the basic steps:
- Sign in to the Azure portal.
- In the left-hand menu, click on "Create a resource".
- In the "New" window, search for "Virtual Machine" and select it.
- In the "Create a Virtual Machine" window, fill in the necessary details like name, region, image, size, etc., and click "Review + create".
- Once the validation is passed, click on "Create".
3. **Handle high load and requests:** If your Azure resource provider is under high load and requests are being throttled, you may need to wait and try again later. If this is a frequent issue, consider spreading your workloads across multiple clusters or adjusting the timing of your jobs to avoid peak usage times.
4. **Ensure your Databricks cluster is not overloaded:** If your cluster has too many jobs running, it can overload the cluster and cause timeouts. As a general rule, move heavier data pipelines to run on their own Azure Databricks clusters.
Remember to continuously monitor your resources and adjust them according to your needs. This will help you use Azure Databricks more seamlessly.
