Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-05-2022 07:55 AM
Hi all,
there is a random error when pushing data from Databricks to a Azure SQL Database.
Anyone else also had this problem? Any ideas are appreciated.
See stacktrace attached.
Target: Azure SQL Database, Standard S6: 400 DTUs
Databricks Cluster config:
"spark_version": "9.1.x-scala2.12",
"spark_conf": {
"spark.driver.extraJavaOptions": "-Dlog4j2.formatMsgNoLookups=true",
"spark.sql.session.timeZone": "UTC",
"spark.driver.maxResultSize": "6g",
"spark.executor.extraJavaOptions": "-Dlog4j2.formatMsgNoLookups=true",
"spark.databricks.io.cache.enabled": "true"
},
"node_type_id": "Standard_E4ds_v4",
"driver_node_type_id": "Standard_E8ds_v4",
Labels:
- Labels:
-
Azure sql database
-
Sqlserver