Hello, everyone. I want to ask if there is a way to connect Databricks cluster with SSH interpreter in your IDE? I know about databricks connect but I want to execute the entire code in the cluster.

[Anonymous]

Nice and amazing post this one is, thanks for sharing…

Subway Live IQ

[Anonymous]

Thanks for sharing, I found a lot of interesting information here. A really good post, very thankful and helpful that you will write many more posts like this one.

MyMileStoneCard

[gbrueckl]

For those of you who are interested, we created an extension for VSCode to work with Databricks which allows you to manage your Databricks workspace (DBFS, Clusters, Secrets, Repos, ...) and also run notebooks interactively against your Databricks Cluster

https://marketplace.visualstudio.com/items?itemName=paiqo.databricks-vscode

It does not require any additional installation (no DBX, no SSH, no specific cluster config, ...) - just your Databricks URL and a Personal Access Token (PAT)

[florent]

can we use an azure ad token instead of the pat?

[gbrueckl]

for our VSCode extension? no, not yet, but thats something we have on our list to add in the near future!

[florent]

yes with vscode extension. Cool for this feature.. the target being not to authorize the creation of a pat for a user.. but to use his azure ad connection. As many Microsoft azure extensions already do.

[bhuvahh]

I think plain python code will run with databricks connect (if it is a python program you are writing), and spark sql can be done by spark.sql(...).