Issue with Non-Bulk Inserts Using JDBC on Databricks Runtime 14.1

cotrariello
Databricks Partner

Hello team,

I am experiencing an issue with insert operations on Databricks using the JDBC driver. In my SAS Viya system, the DatabricksJDBC42.jar driver version 2.6.40 is configured. I’ve noticed that, up to Databricks Runtime version 13.1, insert operations are performed in bulk. However, starting from Runtime version 14.1, these insert operations are executed row by row, leading to a significant slowdown in performance.

I also noticed that with version 13, a log line is displayed (highlighted in yellow in the image on the right), while in version 14 there is no such log entry. Could there be a correlation? Could this difference be relevant?

cotrariello_3-1731582128580.pngcotrariello_2-1731582087923.png

Has anyone else encountered this issue or have any suggestions on how to resolve it? Any advice would be greatly appreciated!

Thank you!