I'm trying to use the SAP JDBC driver to read data in my Spark application which I deploy via asset bundles with job computes.

I was able to install the SAP JDBC Driver on a general purpose cluster by adding the jar (com.sap.cloud.db.jdbc:ngdbc:2.25.9) in the UI via Libraries -> Install new -> maven -> maven coordinates.
Then I could executed the code, no problem.

But when I try to add the jar on a job cluster in Databricks asset bundles, via spark config - like this:

spark_conf:
      "spark.databricks.cluster.profile": "singleNode"
      "spark.master": "local[*]"
      "spark.sql.session.timeZone": "UTC"
      "spark.databricks.dataLineage.enabled": "true"
      "spark.databricks.delta.retentionDurationCheck.enabled": "false"
      "spark.sql.extensions": "io.delta.sql.DeltaSparkSessionExtension"
      "spark.sql.catalog.spark_catalog": "org.apache.spark.sql.delta.catalog.DeltaCatalog"
      "spark.jars.packages": "com.microsoft.azure:azure-storage:8.6.3,org.apache.hadoop:hadoop-azure:3.3.1,io.delta:delta-core_2.12:2.4.0,com.sap.cloud.db.jdbc:ngdbc:2.25.9"

I cannot get it to work, my application fails with the erorr 'java.lang.ClassNotFoundException: com.sap.db.jdbc.Driver' as if it is not installing it correctly. The other jars are available however. 

In the Log4j output of the cluster, I can see the spark config being set correctly:

spark.home=/databricks/spark
spark.jars.packages=com.microsoft.azure:azure-storage:8.6.3,org.apache.hadoop:hadoop-azure:3.3.1,io.delta:delta-core_2.12:2.4.0,com.sap.cloud.db.jdbc:ngdbc:2.25.9

I do not see the sap jar being referenced anywhere else in the log output. 

What am I doing wrong / how can I fix it / furher debug the problem?