Getting python version errors when using pyspark rdd using databricks connect
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-18-2024 04:21 AM - edited 04-18-2024 04:22 AM
Hi community,
When I use pyspark rdd related functions in my environment using databricks connect, I get below error:
Databricks cluster version: 12.2.
`RuntimeError: Python in worker has different version 3.9 than that in driver 3.10, PySpark cannot run with different minor versions. Please check environment variables PYSPARK PYTHON...`
How can I resolve it?
Labels:
- Labels:
-
Databricks connect
-
Databricks Pyspark