Is there a way to control the cluster runtime version for DLT

[Brad]

Hi team, 

When I create a DLT job, is there a way to control the cluster runtime version somewhere? E.g. I want to use 14.3 LTS. I tried to add `"spark_version": "14.3.x-scala2.12",` inside cluster default label but not work.

Thanks

[ezhil]

If you are running as a notebook you can change it while creating the cluster under 
Performance -> Databricks Runtime Version  you can select the needed DBR version.

[Brad]

Thanks. I mean running DLT, not run a cell from notebook sourced for DLT.

[BamBam]

Hi Brad,

I don't think you can because if you look at a recent DBRS article here (that apparently it won't let me link) it says in the "DLT Compute" section: "DLT will manage and optimize the node type and DBR selection, ensuring the best choice of nodes and the latest DBR runtime are selected, reducing the management overhead for users. "

Thanks.

[MinThuraZaw]

Hello @Brad ,

This page listed cluster attributes that you cannot change and why, including spark_version. 

https://docs.databricks.com/en/delta-live-tables/properties.html#cluster-attributes-that-are-not-use...

[Brad]

Thanks. Got it.

And the cluster has to be share mode. Can different DLT jobs share clusters or when DLT job is running, can other people use the cluster? Seems each DLT job running will start a new cluster. If it is not be able to shared, why it has to be share mode?