Runtime Spark 13.2 no longer available

[bedant]

I'm enrolled in the course Get Started with Databricks for Data Engineering

Trying work on one of the demos that asked to copy the code base 

https://labs.training.databricks.com/import/get-started-with-data-engineering-on-databricks/v2.0.3/g...

I tried to run the code under Module 2.3 Demo: Data Management, but was not able to run the first code, under 

Includes/Classroom-Setup-05, cause it calls the code under /Includes/_common. 

That indicates the the DB Spark compute should be 13.2, but Azure no longer has 13.2, it has 13.3 and higher.

How to fix the _common code or any other solution?

[feiyun0112 · Accepted solution]

I think you should change _common code to use actual databricks version

course_config = CourseConfig(course_code = "gswdeod",
                             course_name = "get-started-with-data-engineering-on-databricks",
                             data_source_name = "get-started-with-data-engineering-on-databricks",
                             data_source_version = "v01",
                             install_min_time = "1 min",
                             install_max_time = "5 min",
                             remote_files = remote_files,
                             supported_dbrs = ["13.2.x-scala2.12", "13.2.x-photon-scala2.12", "13.2.x-cpu-ml-scala2.12"],
                             expected_dbrs = "13.2.x-scala2.12, 13.2.x-photon-scala2.12, 13.2.x-cpu-ml-scala2.12")

[bedant]

Yes updated to my exact spark version and now it worked 🙂

supported_dbrs = ["13.3.x-scala2.12", "13.3.x-photon-scala2.12", "13.3.x-cpu-ml-scala2.x"],

expected_dbrs = "13.3.x-scala2.12, 13.3.x-photon-scala2.12, 13.3.x-cpu-ml-scala2.x")