Using spark jars using databricks-connect>=13.0
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-05-2023 04:25 AM
With the newest version of databricks-connect, I cannot configure the extra jars I want to use. In the older version, I did that via
spark = SparkSession.builder.appName('DataFrame').\ config('spark.jars.packages','org.apache.spark:spark-avro_2.12:3.3.0').getOrCreate()
How can I configure this with databricks-connect>=13.0?