cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Spark Version Json code for LTS ML cluster

confusedIntern
New Contributor III

Hi! Any help will be greatly appreciated! Thank you so much!

So I'm currently following this tutorial: https://docs.databricks.com/applications/mlflow/projects.html

In the tutorial, for the cluster specification, it has "spark_version": "7.3.x-scala2.12" in the json code. After I completed the tutorial and run the specified code, I see that the runtime version is 7.3 LTS. I want it to have runtime version 7.3 LTS ML (or specifically 9.1 LTS ML but any LTS ML runtime works). I'm not sure how to accomplish that with the json code.

Please let me know if there is a way! Thank you!

1 ACCEPTED SOLUTION

Accepted Solutions

tomasz
New Contributor III
New Contributor III

You can use the API to get the available runtime versions or the CLI with the command:

databricks clusters spark-versions

View solution in original post

4 REPLIES 4

tomasz
New Contributor III
New Contributor III

You can use the API to get the available runtime versions or the CLI with the command:

databricks clusters spark-versions

Thank you so much!

tomasz
New Contributor III
New Contributor III

Also for 9.1 LTS ML it should be: "9.1.x-cpu-ml-scala2.12"

Yes!! I just figured it out before seeing your post. But thank you so much! I tried it and the cluster is working in 9.1 LTS ML!

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!