topic MLflow Project run always comes back as status failed. in Machine Learning

MLflow Project run always comes back as status failed.

confusedIntern — Wed, 13 Jul 2022 15:18:14 GMT

Hi! This is kind of an urgent question so any help would be greatly appreciated! Thanks so much!

So I'm following this tutorial to try to create an MLflow project: https://docs.databricks.com/applications/mlflow/projects.html

I tried with the example in the tutorial by following into the GitHub repo and downloading the files into my DBFS and creating a cluster-spec json.

This is the code I used:

But this is the result I get back:

I don't understand why I'm getting back status failure when I'm using the same code from the tutorial. And when I go see the experiment, its says there are no artifacts:

Please help! Thank you so much!

Re: MLflow Project run always comes back as status failed.

confusedIntern — Fri, 15 Jul 2022 13:28:46 GMT

Hi @Kaniz Fatma! By new cluster specification, do you mean creating a new cluster using a json file? In the tutorial, this code was what we needed to run in the notebook:

mlflow run <uri> -b databricks --backend-config <json-new-cluster-spec>

so for the <json-new-cluster-spec>, what I have for that code is this: Would this be the new cluster specification?

(Also, the company's databricks I'm using has internet restriction, and I'm not sure if that would factor into this error, but if it does, I would love to have an explanation!)

Thank you so much!!

Re: MLflow Project run always comes back as status failed.

axb0 — Sun, 17 Jul 2022 15:31:06 GMT

https://docs.microsoft.com/en-us/azure/databricks/applications/mlflow/projects

Re: MLflow Project run always comes back as status failed.

confusedIntern — Mon, 18 Jul 2022 13:57:08 GMT

Hi @Alex Barreto and @Kaniz Fatma

I did specify the cluster specifications. It's the .json file from the one line of code. That json file was also uploaded into DBFS for me to use.

I'm still confused as to why this is not working. I have a feeling it might be because my company restricts access to the internet and the project is reaching out to the internet? If so, how do I make it work so it doesn't reach the internet and could work?

Thank you so much!

Re: MLflow Project run always comes back as status failed.

Prabakar — Sun, 24 Jul 2022 20:57:37 GMT

hi @Margie Kale you have the logs for the failed cluster. Check it to have a better understanding of what went wrong.

Re: MLflow Project run always comes back as status failed.

Vidula — Sun, 04 Sep 2022 06:50:42 GMT

Hey there @Margie Kale

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: MLflow Project run always comes back as status failed.

sean_owen — Mon, 19 Sep 2022 22:34:22 GMT

This is generally not how you use MLflow in Databricks. You are already in Databricks so do not need to send code to Databricks to execute. Instead just run your code in a notebook; there is no need to package as an MLflow Project. Projects are primarily for use outside of Databricks, though you can send them to Databricks to execute - though you'd do that from a CLI from elsewhere.