Isi
Honored Contributor III

Hello @lezwon ,

In my opinion, the issue might be related to how you’re using a symlink with a .whl file.

The problem is that pip doesn’t just care about the file name — it also checks the internal metadata of the .whl file, which still declares the version as 0.1.0. When you run the notebook as a job, it might try to resolve package==latest based on the filename, which isn’t a valid version according to Python’s packaging standards (PEP 440), and that could be causing the Invalid requirement error.

It seems like using a symlink in this way might confuse pip during resolution, especially in job contexts.

Maybe you can resolve:

 

Option 1: Install the actual .whl file with its version (no alias)

Instead of creating a symlink like package-latest.whl, simply install the real .whl file with its full version:

If you don’t want to hardcode the version in every notebook, consider parameterizing it using widgets or environment variables when running as a job.

Option 2: Dynamically detect and install the latest wheel

Instead of relying on symlinks, you can upload your .whl files using proper versioning (package-0.1.0.whl, package-0.1.1.whl, etc.), and write a small script that automatically installs the most recent one:

files = dbutils.fs.ls("<path>")

latest = sorted([f for f in files if f.name.startswith("package-") and f.name.endswith(".whl")])[-1].path

%pip install {latest}

(something like that)

This way, you avoid using symlinks or naming tricks, and still always install the latest version available in your volume.

Option 3: Publish

By the way, if you publish your package to a package manager like Nexus or Artifactory, it’s possible that using package==latest could actually work as expected.

These platforms manage version indexes and metadata internally, so they can resolve latest to the most recent published version, even if the filename doesn’t say so. That might help avoid the issues you’re seeing with direct .whl installation via symlinks.

Hope this helps, 🙂

Isi