Cannot configure VS code databricks extension with a non-standard databricks URL: not a databricks host.

[PrawnballNightm]

Hello,

I'm trying to connect to our databricks instance using the vscode extension. However, when following this guide we cannot get the configuration to proceed past the point that it asks for our instance URL.

The prompt appears to expect a URL of the form https://***.cloud.databricks.com where ours is of the form https://*.uk/

We get the error message "not a databricks host" and cannot proceed past this point. If I append the correct suffix to any other string, making an invalid instance URL in the process, I can proceed to the next step.

Is there a workaround to circumvent the plugin incorrectly flagging our URL as invalid? Am I missing something else?

I have tried continuing to configure my .databrickscfg file manually with the correct URL and token, but the profile does not appear as a valid selection in the VS code panel, I believe because of the same URL filter.

Thanks in advance,

Adrian

[Anonymous]

Hi @Adrian Nettleship​ 

Great to meet you, and thanks for your question!

Let's see if your peers in the community have an answer to your question. Thanks.

[Chengcheng]

I don't know how to solve your problem but the URL of my Azure Databricks workspace is also not the form of https://***.cloud.databricks.com but https://adb-***.*.azuredatabricks.net. But the configuration in Visual Studio Databricks extension does work for me.

[Anonymous]

Hi @Adrian Nettleship​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

[PrawnballNightm · Accepted solution]

Hello,

Yes, the databricks team shared a modified version of the vs code plugin which did not include the URL matching logic. It connects successfully. However, our custom URL is as it is because our organisation is hosting its own instance of Databricks within a VPC, behind a corporate firewall. As a result the version of the databricks instance we are running is not sufficiently new to support the vscode plugin features that allow us to connect and execute code on the cluster.

We are moving forward with developing locally within a docker container, rather than with a connection to the databricks instance, and trying to refactor our tests accordingly to run locally. We will also examine databricks-connect as an option to make a connection to our databricks instance for when the tests require it - however, this may also be affected by the limitations related to the version of databricks we are running.

@jamesw's long comment in this forum post appears to be an up to date account of the options and limitations for debugging databricks code: https://community.databricks.com/s/question/0D53f00001HKIByCAP/debugging

Adrian