AutoML on Azure Databricks as of June 2026

[nepiskopos]

Hello everybody,

My team and I are facing a sudden and unexpected issue with training Forecasting models utilizing AutoML on Azure Databricks.

As of two weeks ago, we privatized our Azure Databricks environment behind a private network, accessible only by using a VPN.

Since then, Serverless training of Forecasting ML models fails for every type of forecasting ML framework, including ARIMA, Prophet and DeepAR. To be more precise:

• ARIMA training experiments complete. However, the deployment of trained ARIMA models in Endpoints fails. The respective Service logs are available here.
• Prophet training experiments fail to complete, being cancelled at the Tuning stage. The only output in the run indicating that something is wrong is "databricks.automl.shared.databricks_utils: No host name to create absolute URL".
• DeepAR training experiments also fail to complete, being cancelled at the Tuning stage. The only output in the run indicating that something is wrong is "databricks.automl.shared.databricks_utils: No host name to create absolute URL".

On the other hand, using a compute cluster to train ARIMA and Prophet models, the training completes successfully. The compute runtime is 17.3 LTS for Machine Learning. However, when trying to use these models for inference, we get the following issues.

• For the ARIMA models, the deployment fails, indicating that the failure reason is "mlflowserving.scoring_server.diagnostic_classifier._PhaseTaggedError: [mlflow_parse] ModuleNotFoundError: No module named 'databricks.automl_runtime'"
• For the Prophet models, the deployment completes. However, when trying to use any Prophet model for inference, the procedure fails, indicating that the failure reason is "Inference failed: Encountered an unexpected error while evaluating the model. Verify that the input is compatible with the model for inference. Error ''Prophet' object has no attribute 'stan_backend'' (Support code: INFERENCE_FAILED-9430702a5a9f)"

The serverless training of Forecasting ML models worked perfectly end-to-end a few weeks back and so did the deployment and the inference.

Could we get some help on how to resolve this issues? Is this a temporary bug? Do we need to take any actions?

I have also opened a respective GitHub issue.

Thank you in advance.

[szymon_dybczak]

Hi @nepiskopos ,

I think that your issues all share a single root cause: the restricted egress network policy introduced when you moved behind the private network.

Serverless forecasting requires outbound access to resolve the workspace URL for MLflow metric logging, and to pull packages from PyPI during the Tuning stage. Both are now blocked. Per Databricks docs, if the workspace has serverless egress control enabled, pypi.org must be explicitly added to the Allowed Domains list.

To fix it: In Account Console → Security → Networking → Context-based ingress & egress control, add pypi.org and files.pythonhosted.org to the egress allowlist.  

[iyashk-DB]

Databricks AutoML is still around and actively used as of mid-2026, so nothing's wrong on that front, but there's a nuance worth knowing: starting with Databricks Runtime 18.0 ML, AutoML was removed as a built-in library on classic compute. It's still there, you just install databricks-automl-runtime from PyPI yourself now instead of it being pre-loaded. Serverless forecasting (which is what you're using for ARIMA/Prophet/DeepAR) is a separate thing from that and is still Public Preview and fully maintained, so that part isn't going anywhere.

On the actual failures, the egress fix you got is correct. Serverless forecasting needs outbound access during the tuning stage to pull the forecasting packages from PyPI, and once you lock the workspace behind a private network that access gets cut off, which is exactly what throws "No host name to create absolute URL" for Prophet and DeepAR. Databricks' own docs confirm this directly: if serverless egress control is enabled on the workspace, pypi.org has to be on the allowed domains list, and in practice you'll want files.pythonhosted.org too since that's where pip actually pulls the package files from once it's resolved the index.

The ARIMA case is a bit different since training finishes fine and it's the endpoint deployment that fails. If adding the PyPI domains doesn't fix that one, check the served model's logs for something like numpy.dtype size changed, may indicate binary incompatibility. That's a known issue where the AutoML run logs a pandas version below 2.2.2 alongside a NumPy 2.x environment, and the mismatch breaks at serving time even though it works in a notebook. The fix is to pin numpy in the run's conda.yaml/requirements.txt to a version compatible with the pandas in that environment, then redeploy.

As for the cluster-based (classic compute) runs training fine but inference deployment breaking with module/attribute errors, that's actually expected. One-click model serving deployment from the AutoML UI is only supported for serverless forecasting runs. Classic compute forecasting doesn't wire that up automatically, so you'd need to log the model with its full dependency set yourself and deploy it manually rather than expecting the same one-click flow to work.