Try improving the JDBC URL like
jdbc:databricks://adb-1234567890123456.7.azuredatabricks.net:443/my_schema;
HttpPath=/sql/1.0/endpoints/abcd1234;
AuthMech=3;
UID=myuser;
PWD=mypassword;
Catalog=my_catalog;
Schema=my_schema;
RetryCount=5;
RetryInterval=2000;
On data bricks side
Workload design Separate concurrent ETL jobs into different schemas or catalogs Prevents two jobs from contending for locks on the same schema
SQL design best practice Use fully qualified table names instead of relying on USE Eliminates the need for schema context switching
Monitoring and troubleshooting Track queries causing deadlocks using Databricks system tables Helps identify which workflow/session caused the lock
Cache meta data as best practise
