Connecting to Azure PostgreSQL from Azure Databricks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2023 04:03 AM
Hello,
In Databricks there are 2 ways to connect to PostgreSQL, i.e., using JDBC driver or the named connector as mentioned in the document - https://learn.microsoft.com/en-us/azure/databricks/external-data/postgresql
For JDBC, the driver needs to be installed on the cluster manually, and the named connector is only available from runtime 11.2 onwards.
I wanted to know what are the other differences in the 2 connection methods? is the named connector more performant than JDBC? or is the named connector basically a JDBC driver that comes built-in with 11.2, and hence eliminates the need to install driver manually.
I need to read from and write to Azure PostgreSQL database from Azure Databricks, hence wanted to find out differences from performance perspective.
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-25-2025 04:37 AM
Hi Kp12,
I just wanted to check whether you found the answer or not.
I also want to know the difference because the named connector "PostgreSQL" is overwhelming the CPU of PostgreSQL while inserting 41M rows.