DAB for_each_task python wheel fail
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-11-2025 09:40 AM
using python_wheel_wrapper experimental true allows me to use python_wheel_task on an older cluster.
However, if I embed the python_wheel_task in a for_each_task it fails at runtime with:
"Library installation failed for library due to user error. Error message: Library from /Workspace is not allowed on this cluster. Please switch to using DBR 14.1+ ....."
It seems to me that there is a problem with for_each_task not honoring the experimental feature?
databricks.yml
experimental:
python_wheel_wrapper: true
Task1 - works fine!
python_wheel_task:
package_name: my_package
entry_point: main
libraries:
- whl: ../dist/*.whl
Task 2:
for_each_task:
inputs: "{{tasks.<mystuff>}}"
concurrency: ${var.ingest_concurrency}
task:
task_key: my_key
existing_cluster_id: ${var.cluster_id}
python_wheel_task:
...
# Putting this inside the for_each_task fails on my cluster!
libraries:
- whl: ../dist/*.whl
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-11-2025 12:28 PM
Hi @BillBishop,
I will check on this internally as outcome does not seem to be correct. If possible, upgrade your cluster to DBR 14.1 or later. This would resolve the issue without relying on the experimental feature
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
3 weeks ago
it works as expected in later runtime versions. Thanks, we run with upgraded DBR.

