06-24-2021 03:46 PM
I'm wondering if there's a way to set a monthly budget and have my workloads stop running if I hit it.
06-24-2021 04:17 PM
Currently, we do not have an option to stop the workload based on the DBU usage.
It's possible to set job timeout's to prevent pro-longed execution of jobs. The job timeout can be set to 1.5 times the normal SLA/execution time.
06-25-2021 02:35 PM
Cluster Policies would help with this not only from a cost management perspective but also standardization of resources across the organization as well simplification for a better user experience. You can find Best Practices on leveraging cluster policies - https://docs.databricks.com/administration-guide/clusters/policies-best-practices.html
You can also use combination of chargeback and usage logs to drive good resource usage behavior across the enterprise
never-displayed
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.