Error while establishing JDBC connection to Azure databricks via HTTP proxy
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-02-2022 08:32 AM
I am using the databricks JDBC driver (https://databricks.com/spark/jdbc-drivers-download) to connect to Azure databricks.
The connection needs to be routed through a HTTP proxy. I found parameters that can be configured for using the HTTP proxy:By passing invalid parameters I found that the parameters are parsed by the driver.
Still I see the error:
java.sql.SQLException: [Databricks][DatabricksJDBCDriver](700120) Host ....azuredatabricks.net cannot be resolved through DnsResolver com.databricks.client.jdbc.rpc.InternalDnsResolver. Error Message: No such host is known (....azuredatabricks.net)
at com.databricks.client.jdbc.rpc.CustomDnsResolverLoader.getResolvedHost(Unknown Source)
at com.databricks.client.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
at com.databricks.client.spark.jdbc.DownloadableFetchClientFactory.createClient(Unknown Source)
at com.databricks.client.hivecommon.core.HiveJDBCCommonConnection.connectToServer(Unknown Source)
at com.databricks.client.spark.core.SparkJDBCConnection.connectToServer(Unknown Source)
at com.databricks.client.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source)
at com.databricks.client.spark.core.SparkJDBCConnection.establishConnection(Unknown Source)
at com.databricks.client.jdbc.core.LoginTimeoutConnection.connect(Unknown Source)
at com.databricks.client.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
at com.databricks.client.jdbc.common.AbstractDriver.connect(Unknown Source)
at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:55)
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:355)
at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:115)
at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:665)
at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:544)
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:753)
at cloud.celonis.connector.jdbc.services.DatabaseConnectionService.lambda$getConnection$0(DatabaseConnectionService.java:47)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
So, it seems that the proxy is not used after all.
Do you have any suggestions, what to check and how to make the driver use the proxy? Thank you!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-03-2022 02:22 AM
@Jonas Minning the error message says there is a problem with the DNSresolver. Are you using the default DNS or a custom DNS? Suspect the issue is on the DNS resolution. It would be worth checking with your networking team. If it's a custom DNS, you can try changing it to default and check if this works.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-03-2022 02:55 AM
Thanks for your answer @Prabakar Ammeappin ,
the thing is that I need to use the proxy, so I would expect the Driver to connect to the proxy first, e.g. resolving the hostname of the proxy instead of the hostname of the databricks host.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-05-2022 08:32 PM
We don't think the DNS resolution traffic will go through the Proxy first. Can you try setting below in the host file and see if it connects.
xxxxxxxxxx.azuredatabricks net <IP address>
Ultimately you have to ensure your custom DNS servers are able to resolve the host names without any issues.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-07-2022 12:43 AM
Thanks for the answer @Arvind Ravish,
We added the IP to the local host file.
Then the DNS resolver error disappeared and we got a timeout error instead.
This timeout is probably due to firewall which does not allow direct connection from our client to Azure. So, I think we still are facing the issue that the driver does not seem to use the proxy but tries a direct connection instead.
So, I was wondering which conditions need to be fulfilled for the driver to use the proxy. Do you know something around this?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-02-2022 12:38 AM
Hi @Jonas Minning, were you able to find the solution for this issue? We are also facing similar issue.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-02-2022 01:25 AM
Hi Ravikant,
unfortunately, we have not found another solution than not using a proxy.
Best, Jonas
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-04-2023 05:45 AM
Hi @Jonas Minning , actually I am also having the same issue and when i looked into the driver related documentation I found that the driver currently only supports SOCKS proxies and I believe this is the reason why we are getting this error. So, I wanted to check if there are any other drivers that support HTTP proxies. If anyone has information on such JDBC driver could you please let me know?
The driver that i am currently using is DatabricksJDBC42-2.6.29
Best, Varma