- 5439 Views
- 3 replies
- 2 kudos
Resolved! New spark cluster being configured in local mode
Hi,We have two workspaces on Databricks, prod and dev. On prod, if we create a new all-purpose cluster through the web interface and go to Environment in the the spark UI, the spark.master setting is correctly set to be the host IP. This results in a...
- 5439 Views
- 3 replies
- 2 kudos
- 2 kudos
I found the same issue when choosing the default cluster setup on first setup that when I went to edit the cluster to add an instance profile, I was not able to save without fixing this. Thanks for the tip
- 2 kudos
- 12839 Views
- 15 replies
- 0 kudos
Connecting Databricks Spark Cluster to Postgresql RDS Instance
I am trying to connect my Spark cluster to a Postgresql RDS instance. The Python notebook code that was used is seen below:df = ( spark.read \ .format("jdbc") \ .option("url", "jdbc:postgresql://<connection-string>:5432/database”)\ .option("dbt...
- 12839 Views
- 15 replies
- 0 kudos
- 0 kudos
"Caused by: java.net.SocketTimeoutException: connect timed out" indicate the network connection between Databricks cluster and the postgress database on 5432 port was not established and eventually timed out.As a first step, please ensure the connect...
- 0 kudos