07-13-2022 08:18 AM
Hi! This is kind of an urgent question so any help would be greatly appreciated! Thanks so much!
So I'm following this tutorial to try to create an MLflow project: https://docs.databricks.com/applications/mlflow/projects.html
I tried with the example in the tutorial by following into the GitHub repo and downloading the files into my DBFS and creating a cluster-spec json.
This is the code I used:
But this is the result I get back:
I don't understand why I'm getting back status failure when I'm using the same code from the tutorial. And when I go see the experiment, its says there are no artifacts:
Please help! Thank you so much!
07-15-2022 12:40 AM
Hi @Margie Kale, You must use a new cluster specification when running an MLflow Project on Databricks. Running Projects against existing clusters is not supported.
Check your cluster specifications once and try again.
07-15-2022 06:28 AM
Hi @Kaniz Fatma! By new cluster specification, do you mean creating a new cluster using a json file? In the tutorial, this code was what we needed to run in the notebook:
mlflow run <uri> -b databricks --backend-config <json-new-cluster-spec>
so for the <json-new-cluster-spec>, what I have for that code is this: Would this be the new cluster specification?
(Also, the company's databricks I'm using has internet restriction, and I'm not sure if that would factor into this error, but if it does, I would love to have an explanation!)
Thank you so much!!
07-17-2022 08:31 AM
07-18-2022 06:57 AM
Hi @Alex Barreto and @Kaniz Fatma
I did specify the cluster specifications. It's the .json file from the one line of code. That json file was also uploaded into DBFS for me to use.
I'm still confused as to why this is not working. I have a feeling it might be because my company restricts access to the internet and the project is reaching out to the internet? If so, how do I make it work so it doesn't reach the internet and could work?
Thank you so much!
07-24-2022 01:57 PM
hi @Margie Kale you have the logs for the failed cluster. Check it to have a better understanding of what went wrong.
09-03-2022 11:50 PM
Hey there @Margie Kale
Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.
We'd love to hear from you.
Thanks!
09-19-2022 03:34 PM
This is generally not how you use MLflow in Databricks. You are already in Databricks so do not need to send code to Databricks to execute. Instead just run your code in a notebook; there is no need to package as an MLflow Project. Projects are primarily for use outside of Databricks, though you can send them to Databricks to execute - though you'd do that from a CLI from elsewhere.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group