topic Cannot install wheel from Workspace in DLT in Get Started Discussions

Cannot install wheel from Workspace in DLT

Thms317 — Tue, 22 Oct 2024 08:25:03 GMT

Hi all. I am no longer able to install my custom wheel in my DLT pipeline. No matter what configuration I try I cannot get it to work: parameterized or just hard-coding the path to the wheel. If I run the hard-coded cell with an all-purpose cluster the wheel installs just fine, so I am guessing DLT has not longer access to my workspace files?

I have deployed the pipeline via a Databricks Asset Bundle and I am trying to load from a /Workspace/Shared/ folder. The error message I am getting is always the same:

CalledProcessError: Command 'pip --disable-pip-version-check install Workspace/Shared/.bundle/blifc/acc/files/dist/blifc-*-py3-none-any.whl' returned non-zero exit status 1.

See the attached screenshots for a bunch of things I tried, and the error message.

Re: Cannot install wheel from Workspace in DLT

Thms317 — Tue, 22 Oct 2024 14:11:05 GMT

I managed to fix the issue. The problem was that my wheel was built for Databricks Runtime 14.3 LTS and I was using the PREVIEW channel rather than the CURRENT channel. At time of writing:

CURRENT(default): Databricks Runtime 14.1 --> Python: 3.10.12
PREVIEW: Databricks Runtime 15.2 --> Python: 3.11.0

I am not very happy about the generic error message, so hopefully Databricks will fix that in the future.

Re: Cannot install wheel from Workspace in DLT

bhanu_gautam — Wed, 30 Oct 2024 07:36:55 GMT

@Thms317 ,Thanks for sharing