cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Connecting to Azure PostgreSQL from Azure Databricks

kp12
New Contributor II

Hello,

In Databricks there are 2 ways to connect to PostgreSQL, i.e., using JDBC driver or the named connector as mentioned in the document -  https://learn.microsoft.com/en-us/azure/databricks/external-data/postgresql

For JDBC, the driver needs to be installed on the cluster manually, and the named connector is only available from runtime 11.2 onwards.

I wanted to know what are the other differences in the 2 connection methods? is the named connector more performant than JDBC? or is the named connector basically a JDBC driver that comes built-in with 11.2, and hence eliminates the need to install driver manually.

I need to read from and write to Azure PostgreSQL database from Azure Databricks, hence wanted to find out differences from performance perspective.

Thanks.

1 REPLY 1

sharukh_lodhi
New Contributor III

Hi Kp12,
I just wanted to check whether you found the answer or not.
I also want to know the difference because the named connector "PostgreSQL" is overwhelming the CPU of PostgreSQL while inserting 41M rows.

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now