Databricks Connect Vscode. Cannot find package installed on cluster

brady_tyson — Tue, 03 Sep 2024 20:59:05 GMT

I am using Databricks Connect v2 to connect to a UC enabled cluster. I have a package I have made and installed in a wheel file on the cluster.

When using vscode to import the package and use it I get a module not found error when running cell by cell or debugging.

How can I use the package in my local development environment with databricks connect?

Re: Databricks Connect Vscode. Cannot find package installed on cluster

VZLA — Mon, 23 Dec 2024 16:58:44 GMT

Hi @brady_tyson just checking—are you still facing this issue with using your custom package and Databricks Connect? If so, here are a few questions to collect some data points about your setup:

Is Databricks Connect properly installed and configured in your local environment?
Is the package also installed locally?
Is the Python path in your local environment updated to include the package's directory?
Are you using the Databricks extension in Visual Studio Code for built-in support with Databricks Connect?
When running or debugging in VS Code, is the Databricks Connect configuration active to execute code on the cluster?

For now I would say install the same wheel file locally so that your development environment can import the package. Then Databricks Connect will find and use it when you run or debug code in VS Code. I couldn't find this explicitly mentioned, more like implicit. If you have solve this already, could you please share your solution? Greatly appreciated.

topic Re: Databricks Connect Vscode. Cannot find package installed on cluster in Data Engineering

Databricks Connect Vscode. Cannot find package installed on cluster

Re: Databricks Connect Vscode. Cannot find package installed on cluster