Nivethan_Venkat
Databricks MVP

Hi @lukasz_wybieral

It is not necessary to specify the cluster_config, if you would like to use serverless. Be default, Databricks picks the Serverless cluster if you don't specify the cluster configuration. Attaching below databricks.yml for your reference:

Nivethan_Venkat_0-1754990653880.png

This is my how my job looks like if the cluster configuration is not specified:

Nivethan_Venkat_1-1754990723131.png

But it is good to refer here, for which jobs Serverless computes can be configured: https://docs.databricks.com/aws/en/jobs/compute


Thanks & Regards,
Nivethan V