How are .whl files executed for Python wheel tasks?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-27-2025 09:11 AM
Hello,
We package a Poetry managed project into a .whl and run it as a Python wheel task. Naturally, many of the dependencies referenced by the .whl file are already present on the Databricks cluster. Is this detected by the task setup (in its virtual env I suppose) and those dependencies are not pulled from a repo or will the dependencies be re-installed regardless? If so, is there a way to avoid this extra overhead?
Thank you,
David