If you want instant processing, you will have to have a cluster running all the time.

As mentioned above, Databricks is testing serverless compute for data engineering workloads (comparable to serverless SQL). This fires up a cluster in a few seconds, but is not yet GA.

The closest you can get at the moment is cluster pools.