12-01-2021 06:39 AM
Hi,
I am trying to follow this simple document to be able to run MLFlow within Databricks:
https://docs.microsoft.com/en-us/azure/databricks/applications/mlflow/projects
I try to run it from:
I have tested that the 3 methods are properly set-up. I get the same error with all methods:
mlflow.exceptions.RestException: BAD_REQUEST: Unable to connect to the linked AzureML workspace. Check that the workspace exists.
The Databricks workspace is linked to an AzureML workspace.
By following this other document:
https://docs.microsoft.com/en-us/azure/machine-learning/how-to-use-mlflow-azure-databricks
I am actually able to create and run MLFlow experiments from:
The issue that I have is that:
If I follow this document:
https://docs.microsoft.com/en-us/azure/databricks/applications/mlflow/access-hosted-tracking-server
I am able to use the remote mlflow-cli and I can for example, create an experiment in databricks only (the experiment doesn't live in AzureML), by use of:
mlflow experiments create -n /Users/<your-username>/my-experiment
But again, when trying to do something like this:
mlflow run https://github.com/mlflow/mlflow#examples/sklearn_elasticnet_wine -b databricks --backend-config cluster-spec.json --experiment-id <experiment-id>
I get the error I previously mentioned:
mlflow.exceptions.RestException: BAD_REQUEST: Unable to connect to the linked AzureML workspace. Check that the workspace exists.
I have set
export MLFLOW_TRACKING_URI=databricks
And everything else as noted in the documentation.
Is there a configuration I am missing?
Thanks a lot in advance for any help!
12-01-2021 07:12 AM
Maybe this answer will help:
https://community.databricks.com/s/question/0D53f00001UOu7rCAD/mlflow-resourcealreadyexists
as @Prabakar Ammeappin wrote " it’s not recommended to “link” the Databricks and AML workspaces, as we are seeing more problems"
12-01-2021 07:12 AM
Maybe this answer will help:
https://community.databricks.com/s/question/0D53f00001UOu7rCAD/mlflow-resourcealreadyexists
as @Prabakar Ammeappin wrote " it’s not recommended to “link” the Databricks and AML workspaces, as we are seeing more problems"
12-01-2021 07:51 AM
Hi,
Thanks a lot for the help! This linking within WS was already set by somebody working in the team. I will investigate the reason why and try to unlink them. I will report back and say if this suits the trick as the error is not so well documented and it might help others.
12-01-2021 08:25 AM
@Arturo Amador - That would be great! Once you share your solution, would you be happy to mark your answer as best so others can find it more easily?
12-01-2021 08:13 AM
Hi @Hubert Dudek Thanks for sharing the post.
12-01-2021 08:27 AM
you are welcome 🙂
12-15-2021 02:11 AM
Hi! Thanks a lot for your help. I took contact with tech support via my Azure subscription. They confirmed that indeed, connecting the Databricks and AzureML workspaces introduces errors. They provided us with a ARM template for removing the connection. After the connection was successfully removed, there were no more problems with tracking experiments with MLFlow directly in the Databricks WS.
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.