Where can I find documentation on how to set cluster settings (e.g., AWS instance type, spot vs on-demand, number of machines) in Databricks Asset Bundle databicks.yaml files? The only documentation I've come across mentions these things indirectly, for example this link.
Here is an example part my DAB that I'm hoping to complete:
resources:
jobs:
clicks:
job_clusters:
- job_cluster_key: job_cluster
new_cluster:
node_type_id: i3.xlarge
spark_version: 13.3.x-scala2.12
But I'm not sure the names of fields to specify spot vs on demand, number of machines, etc. I'm having a really hard time finding comprehensive documentation on DABs and their options on Google and on this site.