Anonymous
Not applicable

@Alexander Wagner​ :

Databricks Delta Lake supports bulk writes, which are more efficient than row-wise writes. However, the method for achieving this may vary depending on the specific JDBC driver being used. Here are a few possible solutions:

  1. Use the Apache Spark JDBC connector instead of the Databricks JDBC connector. This connector supports bulk inserts, which can be used with Delta Lake. You can specify the batch size using the batchsize option when creating the JDBC connection.
  2. Use the spark.databricks.delta.maxFileSize option to control the size of the files being written. By setting a large value for this option, you can ensure that Delta Lake will write larger files and improve write performance.
  3. Use the OPTIMIZE command to merge small files into larger files. This will improve query performance and reduce storage overhead. You can run this command periodically on your Delta tables to optimize them.
  4. If you are using the Databricks JDBC connector, you can try setting the rewriteBatchedStatements option to true. This can improve write performance by allowing the driver to send multiple statements in a single batch.

Keep in mind that the optimal solution may vary depending on your specific use case, so it's important to test and evaluate the performance of each approach.