cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

display() function always return connection refused on tunneling despite successfully retrieving the schema

antoooks
New Contributor III

Hi everyone,

I am using SSH tunnelling with SSHTunnelForwarder to reach a target AWS RDS PostgreSQL database. The connection got through, however when I tried to display the retrieved data frame it always throws "connection refused" error. Please see screenshot below for clarity. I don't understand how this function works, it seems like it does not reuse the established connection from sparks.read command. Can somebody help me on this?

What I already tried:

  • SSH into the cluster, try tunnelling, and psql from there [works]
  • directly connect to db instance from proxy host [works]
  • Using panda to load the data frame instead of sparks [does not work]
  • Create new SSH key pair for Databricks [does not work]
  • Change postgresql.conf listen_address to ''*"

Screenshot:

image.png 

Thanks.

Regards,

Kurnianto

1 ACCEPTED SOLUTION

Accepted Solutions

jose_gonzalez
Moderator
Moderator

hi @Kurnianto Trilaksono Sutjipto​ ,

This seems like a connectivity issue with the url you are trying to connect to. It fails during the display() command because read is a lazy transformation and it will not be executed right away. On the other hand, display() is an action and it will trigger the lazy transformation.

The following doc will help you to check if you can connect tho this host docs

View solution in original post

3 REPLIES 3

Kaniz
Community Manager
Community Manager

Hi @ antoooks! My name is Kaniz, and I'm the technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else I will get back to you soon. Thanks.

jose_gonzalez
Moderator
Moderator

hi @Kurnianto Trilaksono Sutjipto​ ,

This seems like a connectivity issue with the url you are trying to connect to. It fails during the display() command because read is a lazy transformation and it will not be executed right away. On the other hand, display() is an action and it will trigger the lazy transformation.

The following doc will help you to check if you can connect tho this host docs

If you would like to check for connectivity from databricks to your host, please execute the following command in your notebook.

%sh nc -vz <jdbcHostname> <jdbcPort>

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.