Spark Version Json code for LTS ML cluster

[confusedIntern]

Hi! Any help will be greatly appreciated! Thank you so much!

So I'm currently following this tutorial: https://docs.databricks.com/applications/mlflow/projects.html

In the tutorial, for the cluster specification, it has "spark_version": "7.3.x-scala2.12" in the json code. After I completed the tutorial and run the specified code, I see that the runtime version is 7.3 LTS. I want it to have runtime version 7.3 LTS ML (or specifically 9.1 LTS ML but any LTS ML runtime works). I'm not sure how to accomplish that with the json code.

Please let me know if there is a way! Thank you!

[tomasz · Accepted solution]

You can use the API to get the available runtime versions or the CLI with the command:

databricks clusters spark-versions

[confusedIntern]

Thank you so much!

[tomasz]

Also for 9.1 LTS ML it should be: "9.1.x-cpu-ml-scala2.12"

[confusedIntern]

Yes!! I just figured it out before seeing your post. But thank you so much! I tried it and the cluster is working in 9.1 LTS ML!