Hi team,
New to Databricks and trying to understand if there is a "True" auto-start capability with Databricks. We are evaluating Databricks Delta lake as an alternative cloud based datawarehouse solution but the biggest problem I see is the inability to allow a cluster to auto-start.
Even setting up a pool of idle VMs slightly improves the start up time for clusters of different sizes but there doesn't seem to be a way to auto-start a cluster upon a jdbc request from an external client. (IE business user making a BI request)
What I have seen in the community is references to start up scripts which are scheduled which would be fine for known workloads but I'm referring to a scenario where the usage is unknown. So needs to be automated.
Let me know if there is something obvious I'm missing but can't seem to see a solution with Databricks which means it will sit back in the Datascience/ML environment and leaving other Cloud Based Datawarehouses to run the BI workloads.
