Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-03-2025 12:47 PM
Here is some information to consider:
Databricks Connect currently does not fully support Scala 2.13. The support for Scala 2.13 in Databricks Connect is planned for its version associated with DBR 17.0, expected in early June 2025. Databricks Connect's dependencies and compatibility requirements necessitate careful version alignment between Scala versions used in development and runtime configurations.
For alternative solutions to running your project with Scala 2.13 on DBR 16.4, you can consider the following approaches:
-
Use Direct Cluster Execution: Since DBR 16.4 supports both Scala 2.12 and 2.13, you can deploy jobs directly on the cluster using the Scala 2.13 runtime.
-
Address Dependency Conflicts Manually: As suggested in migration guides, updating build files to resolve dependency conflicts between Scala 2.12 and 2.13 may involve manual adjustments such as excluding incompatible libraries or adding library shading techniques.
-
Cross-Build Projects: If immediate Databricks Connect compatibility is crucial, cross-compile your code with Scala 2.12 to maintain compatibility while leveraging DBR 16.4's flexibility to run Scala 2.13-based tests.
These measures can help mitigate dependency management issues and reduce manual exclusions until full support for Scala 2.13 is released in Databricks Connect.
Cheers, Lou.
