Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-19-2024 11:52 AM
I'm enrolled in the course Get Started with Databricks for Data Engineering
Trying work on one of the demos that asked to copy the code base
I tried to run the code under Module 2.3 Demo: Data Management, but was not able to run the first code, under
Includes/Classroom-Setup-05, cause it calls the code under /Includes/_common.
That indicates the the DB Spark compute should be 13.2, but Azure no longer has 13.2, it has 13.3 and higher.
How to fix the _common code or any other solution?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-19-2024 11:21 PM
I think you should change _common code to use actual databricks version
course_config = CourseConfig(course_code = "gswdeod",
course_name = "get-started-with-data-engineering-on-databricks",
data_source_name = "get-started-with-data-engineering-on-databricks",
data_source_version = "v01",
install_min_time = "1 min",
install_max_time = "5 min",
remote_files = remote_files,
supported_dbrs = ["13.2.x-scala2.12", "13.2.x-photon-scala2.12", "13.2.x-cpu-ml-scala2.12"],
expected_dbrs = "13.2.x-scala2.12, 13.2.x-photon-scala2.12, 13.2.x-cpu-ml-scala2.12")Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-26-2024 09:26 AM
Yes updated to my exact spark version and now it worked 🙂
supported_dbrs = ["13.3.x-scala2.12", "13.3.x-photon-scala2.12", "13.3.x-cpu-ml-scala2.x"],
expected_dbrs = "13.3.x-scala2.12, 13.3.x-photon-scala2.12, 13.3.x-cpu-ml-scala2.x")
