Connecting to Azure PostgreSQL from Azure Databricks

kp12 — Tue, 01 Aug 2023 11:03:39 GMT

Hello,

In Databricks there are 2 ways to connect to PostgreSQL, i.e., using JDBC driver or the named connector as mentioned in the document - https://learn.microsoft.com/en-us/azure/databricks/external-data/postgresql

For JDBC, the driver needs to be installed on the cluster manually, and the named connector is only available from runtime 11.2 onwards.

I wanted to know what are the other differences in the 2 connection methods? is the named connector more performant than JDBC? or is the named connector basically a JDBC driver that comes built-in with 11.2, and hence eliminates the need to install driver manually.

I need to read from and write to Azure PostgreSQL database from Azure Databricks, hence wanted to find out differences from performance perspective.

Thanks.

Re: Connecting to Azure PostgreSQL from Azure Databricks

sharukh_lodhi — Tue, 25 Mar 2025 11:37:20 GMT

Hi Kp12,
I just wanted to check whether you found the answer or not.
I also want to know the difference because the named connector "PostgreSQL" is overwhelming the CPU of PostgreSQL while inserting 41M rows.

topic Re: Connecting to Azure PostgreSQL from Azure Databricks in Data Engineering

Connecting to Azure PostgreSQL from Azure Databricks

Re: Connecting to Azure PostgreSQL from Azure Databricks