cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
supremefist
by • New Contributor III
  • 3293 Views
  • 5 replies
  • 2 kudos

Resolved! New spark cluster being configured in local mode

Hi,We have two workspaces on Databricks, prod and dev. On prod, if we create a new all-purpose cluster through the web interface and go to Environment in the the spark UI, the spark.master setting is correctly set to be the host IP. This results in a...

Data Engineering
Reply
  • 3293 Views
  • 5 replies
  • 2 kudos
View Replies
Latest Reply
scottb
New Contributor II
  • 2 kudos

I found the same issue when choosing the default cluster setup on first setup that when I went to edit the cluster to add an instance profile, I was not able to save without fixing this. Thanks for the tip

  • 2 kudos
4 More Replies
Cano
by • New Contributor III
  • 7610 Views
  • 15 replies
  • 0 kudos

Connecting Databricks Spark Cluster to Postgresql RDS Instance

I am trying to connect my Spark cluster to a Postgresql RDS instance. The Python notebook code that was used is seen below:df = ( spark.read \ .format("jdbc") \ .option("url", "jdbc:postgresql://<connection-string>:5432/database”)\ .option("dbt...

Data Engineering
Reply
  • 7610 Views
  • 15 replies
  • 0 kudos
View Replies
Latest Reply
User16873043099
Contributor
  • 0 kudos

"Caused by: java.net.SocketTimeoutException: connect timed out" indicate the network connection between Databricks cluster and the postgress database on 5432 port was not established and eventually timed out.As a first step, please ensure the connect...

  • 0 kudos
14 More Replies
Labels