Workflow entry point not working on runtime 16.4-LTS

[mexcram]

Hello all,

I'm developing a python code that it is packaged as a wheel and installed inside a docker image. However, since this program requires numpy >= 2.0 I'm forced to use the runtime 16.4-LTS.

When I try to run it as a workflow on databricks I'm encountering the following issue:

Now, what it is odd, is that this exactly workflow is working on a runtime 15.4-LTS (but I have to remove my numpy code).

Do you know if something has changed on the way we have to configure the workflows on runtime 16.4-LTS ?

PS: just for information the issue with numpy >= 2.0 on runtime 15.4-LTS is this:

[Nishant1994]

Facing same issue . 

[mark_ott]

Databricks Runtime 16.4-LTS introduced some changes that affect workflow configuration, environment management, and, notably, support for Python third-party libraries like NumPy. The main issue you’re encountering with NumPy >= 2.0 in Databricks workflows stems from compatibility and environment management changes in this latest LTS release.

What's Different in 16.4-LTS Workflows

• Databricks Runtime 16.4-LTS ships with NumPy 1.26.4 by default. If your workflow (or a third-party wheel) requires NumPy 2.x and tries to upgrade it, you can end up with a version mismatch between NumPy and other libraries like pandas or PySpark, which do not yet support NumPy 2.x fully in Databricks clusters.​

• Databricks clusters (including custom Docker images) may not automatically reconcile these dependency conflicts, especially when the workflow installs packages that force NumPy to 2.x, breaking compatibility with Databricks-native library expectations.

• Databricks Connect and certain libraries explicitly restrict NumPy versions, usually supporting only NumPy < 2.0 due to upstream compatibility issues in PySpark.​

• Python package management in 16.4-LTS (and especially Machine Learning runtimes) is done with virtualenv, which isolates installations but can still end up with incompatible versions if dependencies are upgraded within a workflow or notebook.​

Known numpy 2.0 Issues

• NumPy 2.0 includes breaking changes for both Python and C APIs. Packages not rebuilt against NumPy 2.0 may throw errors like “module 'numpy._core' has no attribute 'multiarray'” or binary incompatibility warnings.​

• Some Databricks tools—including Databricks Connect and possibly workflow orchestrators—still require NumPy < 2.0 for reliable operation and explicitly block or downgrade the package.​

Recommendations

• On Databricks Runtime 16.4-LTS, pin NumPy to the version provided by the runtime, unless absolutely necessary. That means in most cases you should stick to numpy==1.26.4.

• If you must use NumPy 2.x, ensure all dependencies, including pandas (version >=2.2.2), fully support it, and test workflows in isolated environments.​

• When building wheels for deployment on Databricks, specify compatible versions in your requirements. Adding explicit constraints (e.g., numpy==1.26.4) avoids most runtime errors on 16.4-LTS.​

• When using custom Docker images, ensure the pre-installed libraries do not conflict with Databricks-managed base libraries. If you force a newer numpy, check all downstream package versions and compatibility carefully.

Key Table: Databricks Runtime, NumPy Support, and Workflow Guidance

Runtime Version	Default NumPy	Supports NumPy 2.x?	Guidance
15.4-LTS	1.26.x	No	Remove NumPy 2.x code; stay <2.0
16.4-LTS	1.26.4	No*	Pin numpy==1.26.4, avoid 2.x if possible
Custom Image	User choice	Possible, risky	Must align all deps; not guaranteed

*16.4-LTS clusters may allow pip installing NumPy 2.x, but this breaks compatibility with Spark and some Databricks tools.​