cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
supremefist
by • New Contributor III
  • 6266 Views
  • 3 replies
  • 2 kudos

Resolved! New spark cluster being configured in local mode

Hi,We have two workspaces on Databricks, prod and dev. On prod, if we create a new all-purpose cluster through the web interface and go to Environment in the the spark UI, the spark.master setting is correctly set to be the host IP. This results in a...

Data Engineering
Reply
  • 6266 Views
  • 3 replies
  • 2 kudos
View Replies
Latest Reply
scottb
New Contributor II
  • 2 kudos

I found the same issue when choosing the default cluster setup on first setup that when I went to edit the cluster to add an instance profile, I was not able to save without fixing this. Thanks for the tip

  • 2 kudos
2 More Replies
Cano
by • New Contributor III
  • 14355 Views
  • 15 replies
  • 0 kudos

Connecting Databricks Spark Cluster to Postgresql RDS Instance

I am trying to connect my Spark cluster to a Postgresql RDS instance. The Python notebook code that was used is seen below:df = ( spark.read \ .format("jdbc") \ .option("url", "jdbc:postgresql://<connection-string>:5432/database”)\ .option("dbt...

Data Engineering
Reply
  • 14355 Views
  • 15 replies
  • 0 kudos
View Replies
Latest Reply
User16873043099
Contributor
  • 0 kudos

"Caused by: java.net.SocketTimeoutException: connect timed out" indicate the network connection between Databricks cluster and the postgress database on 5432 port was not established and eventually timed out.As a first step, please ensure the connect...

  • 0 kudos
14 More Replies
Labels
Top