This means the network is fine, but something in the spark config is amiss.

What are the DBR version and the hive version? Please check f you are using a compatible version.

If you don't specify any version, it will take 1.3 and you wouldn't have to use jars in this case.

Please use below config with your appropriate values,

spark.hadoop.datanucleus.fixedDatastore false
 
spark.hadoop.datanucleus.autoCreateSchema true
 
spark.hadoop.datanucleus.autoCreateTables true
 
spark.hadoop.javax.jdo.option.ConnectionURL jdbc:mariadb://<database-name>.cj11tymkwz5w.us-west-2.rds.amazonaws.com
 
:3306/databricks_metastore?createDatabaseIfNotExist=true
 
spark.hadoop.javax.jdo.option.ConnectionDriverName org.mariadb.jdbc.Driver
 
spark.hadoop.javax.jdo.option.ConnectionUserName <value> 
 
spark.hadoop.javax.jdo.option.ConnectionPassword <value>