cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Using Databricks Connect with PyCharm behind Proxy

ADuma
New Contributor III

I am using Databricks connect with PyCharm in order to locally develop my pipelines before deploying to Databricks. I set up authentication for databricks-connect using the following guide https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect/python/pycharm

This is working well when I'm working with a normal internet connection. But when I'm working from my office I enter through a company network which prevents access to many services. I can get around this by specifying the HTTPS_PROXY environment variable for many services (git, pip etc.)

But I don't manage to connect to my Databricks cluster through databricks-connect. I just get the following error message:

 

ValueError: default auth: databricks-cli: cannot get access token: Error: oidc: fetch .well-known: Get "https://my_workspace.azuredatabricks.net/oidc/.well-known/oauth-authorization-server": proxyconnect tcp: tls: first record does not look like a TLS handshake. Config: host=https://my_workspace.azuredatabricks.net, profile=my-profile, auth_type=databricks-cli, cluster_id=my_cluster_id

 

Does anyone know how to set the proxy config for databricks-connect? Do I need to set some values in .databrickscfg?

1 ACCEPTED SOLUTION

Accepted Solutions

ADuma
New Contributor III

I managed to solve the problem by including my login and password in the HTTPS_PROXY environment variable.

I set it to something like this HTTPS_PROXY = "http://username:password@proxy.company.com:port" and databricks-connect can create spark sessions in my pycharm now.

View solution in original post

1 REPLY 1

ADuma
New Contributor III

I managed to solve the problem by including my login and password in the HTTPS_PROXY environment variable.

I set it to something like this HTTPS_PROXY = "http://username:password@proxy.company.com:port" and databricks-connect can create spark sessions in my pycharm now.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group