topic Re: Not Able to run AutoML - RESOURCE DOES NOT EXIST ERROR in Machine Learning

Not Able to run AutoML - RESOURCE DOES NOT EXIST ERROR

nt2 — Mon, 14 Jul 2025 11:10:14 GMT

Hello,

I'm new to both ML and Databricks. I'm running a Classification Experiment and getting a RESOURCE DOES NOT EXIST ERROR. It says the experiment_id does not exist. Can you help me point where to fix the error? I tried the Diagnose Error option, but I get the same result. Thanks in advance.

Re: Not Able to run AutoML - RESOURCE DOES NOT EXIST ERROR

Khaja_Zaffer — Mon, 14 Jul 2025 11:34:01 GMT

Hello Nt2

good day!!

If you view a stack trace and it looks similar to the following:

RestException Traceback (most recent call last)
File <command-XXXXXXXXXXXX>:7
2 mlflow.sklearn.autolog()
...

File /databricks/python/lib/python3.9/site-packages/mlflow/tracking/fluent.py:349, in start_run(run_id, experiment_id, run_name, nested, tags, description)
345 user_specified_tags[MLFLOW_RUN_NAME] = run_name
347 resolved_tags = context_registry.resolve_tags(user_specified_tags)
--> 349 active_run_obj = client.create_run(
350 experiment_id=exp_id_for_run, tags=resolved_tags, run_name=run_name
351 )
353 _active_run_stack.append(ActiveRun(active_run_obj))
354 return _active_run_stack[-1]

I did the repo for you.

And to resolve the issue:

You should validate that the service principal has the necessary edit permissions.

By granting edit permissions to the service principal, the job using the client.create_run method (the client is an instance of MlflowClient) can run successfully.

This edit permission ensures the necessary resources, such as the node with the specified ID, are accessible and available for the MLflow operation.

Re: Not Able to run AutoML - RESOURCE DOES NOT EXIST ERROR

nt2 — Mon, 14 Jul 2025 12:49:48 GMT

I appreciate the detailed explanation @Khaja_Zaffer !

Re: Not Able to run AutoML - RESOURCE DOES NOT EXIST ERROR

AIChief — Wed, 23 Jul 2025 04:41:41 GMT

appreciated