cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
mrmota
by • New Contributor
  • 1746 Views
  • 1 replies
  • 3 kudos

ssh connection with paramiko

I'm trying to use the notebook with python to access our server through ssh. But I am not able to connect through the paramiko library. I have already authorized our server's firewall to accept the databrick ip. Have you ever had a similar case? can ...

Data Engineering
Reply
  • 1746 Views
  • 1 replies
  • 3 kudos
View Replies
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 3 kudos

yes this can happen you have to whitelist that Port in the databricks configuration, there is one spark configuration that you can add to your cluster, that will whitelist your particular portas that configuration can only give by databricks, you can...

  • 3 kudos
BorislavBlagoev
by • Valued Contributor III
  • 27551 Views
  • 33 replies
  • 14 kudos

Hello, everyone. I want to ask if there is a way to connect Databricks cluster with SSH interpreter in your IDE? I know about databricks connect but I want to execute the entire code in the cluster.

Data Engineering
Reply
  • 27551 Views
  • 33 replies
  • 14 kudos
View Replies
Latest Reply
bhuvahh
New Contributor II
  • 14 kudos

I think plain python code will run with databricks connect (if it is a python program you are writing), and spark sql can be done by spark.sql(...).

  • 14 kudos
32 More Replies
antoooks
by • New Contributor III
  • 2630 Views
  • 2 replies
  • 4 kudos

Resolved! display() function always return connection refused on tunneling despite successfully retrieving the schema

Hi everyone,I am using SSH tunnelling with SSHTunnelForwarder to reach a target AWS RDS PostgreSQL database. The connection got through, however when I tried to display the retrieved data frame it always throws "connection refused" error. Please see ...

image.png
Data Engineering
Reply
  • 2630 Views
  • 2 replies
  • 4 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 4 kudos

hi @Kurnianto Trilaksono Sutjipto​ ,This seems like a connectivity issue with the url you are trying to connect to. It fails during the display() command because read is a lazy transformation and it will not be executed right away. On the other hand,...

  • 4 kudos
1 More Replies
Zen
by • New Contributor III
  • 3837 Views
  • 2 replies
  • 3 kudos

Resolved! ssh onto Cluster as root

Hello, I'm following the instructions here:https://docs.databricks.com/clusters/configure.html?_ga=2.17611385.1712747127.1631209439-1615211488.1629573963#ssh-access-to-clustersto ssh onto the Driver node, and it's working perfectly when I ssh on as `...

Data Engineering
Reply
  • 3837 Views
  • 2 replies
  • 3 kudos
View Replies
Latest Reply
cconnell
Contributor II
  • 3 kudos

I am 99% sure that logging into a Databricks node as root will not be allowed.

  • 3 kudos
1 More Replies
MoJaMa
by • Databricks Employee
  • 961 Views
  • 1 replies
  • 1 kudos

Does Databricks support SSH at all for Repos? We only see HTTPS in the UI.

Data Engineering
Reply
  • 961 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
MoJaMa
Databricks Employee
  • 1 kudos

Only HTTPS is supported right now.If SSH is required for your use case, please let your Databricks Rep know and reference the Idea DB-I-3697 so that it can be prioritized.

  • 1 kudos
Labels