topic Re: MLflow: Connect python with Community Edition in Machine Learning

MLflow: Connect python with Community Edition

nicobuko — Wed, 11 Dec 2024 20:07:07 GMT

Hello,

I am new to databricks and want to work with MLFlow in the Databricks Community Edition. In python i am using mlflow.login(). This requests me to enter a password. But i do not have any password due to the fact that databricks login only required a one time password.

Is there another way to connect python Code to the community Edition? Or how do i create a password?

In the settings in the community Edition there is a password changing functionality but this one Requires to enter the old password which obviously does not exist.

It would be great if someone could Provide some help! Thank you very much!

Re: MLflow: Connect python with Community Edition

Walter_C — Wed, 11 Dec 2024 20:59:49 GMT

You can try the following:

Instead of using mlflow.login(), you can set the tracking URI and the token directly in your code. Here is an example:

import mlflow # Set the tracking URI to your Databricks workspace mlflow.set_tracking_uri("https://community.cloud.databricks.com") # Set the Databricks token token = "YOUR_PERSONAL_ACCESS_TOKEN" # Configure the MLFlow client to use the token mlflow.set_experiment("your-experiment-name") mlflow.start_run() mlflow.log_param("param1", 5) mlflow.log_metric("metric1", 0.85) mlflow.end_run()

Re: MLflow: Connect python with Community Edition

nicobuko — Wed, 11 Dec 2024 21:05:10 GMT

@Walter_C Thank you for your fast answer 🙂

Where can I generate such a token in the community edition? I did not see an option to generate a token when I have looked into the community Edition.

Re: MLflow: Connect python with Community Edition

Walter_C — Wed, 11 Dec 2024 21:42:13 GMT

Under Settings > Developer it does not allow you to?

Re: MLflow: Connect python with Community Edition

Rjdudley — Thu, 12 Dec 2024 14:19:57 GMT

The Community Edition is very feature limited. PATs are one feature which is not available. None of the SQL Warehouse features are available, nor are most of the AI/ML features.

Re: MLflow: Connect python with Community Edition

Walter_C — Thu, 12 Dec 2024 14:47:44 GMT

Got it, as per our docs it is suggested the creation of a token for Community edition as you can see here https://docs.databricks.com/en/mlflow/access-hosted-tracking-server.html#step-1-configure-your-environment it states that for Community Edition you can use any of the two approaches listed there, can you go through the doc and let me know if this are the methods you have attempted?

Re: MLflow: Connect python with Community Edition

nicobuko — Fri, 13 Dec 2024 07:58:17 GMT

@Rjdudley Thank you for your answer but I am wondering about your answer because the mlflow documentation explicitly says that the CE could be used: https://mlflow.org/docs/latest/getting-started/running-notebooks/index.html

But the way described in the docs via mlflow.login is obviously not functional because i do not have a password.

@Walter_C Before your comment i had seen the doc that you have linked once. But both Suggestions there are relying on the need to have a password which a User obviously does not have because of one time passwords. So the doc seems to be outdated and for me there seems to be currently no way to connect...

Re: MLflow: Connect python with Community Edition

Advika_ — Fri, 13 Dec 2024 08:47:09 GMT

Hello @nicobuko!

You can find helpful resources for Databricks Community Edition here. If the available resource doesn’t resolve your concern, feel free to submit a ticket with Databricks Support team for further assistance.

Thank you.

Re: MLflow: Connect python with Community Edition

Rjdudley — Fri, 13 Dec 2024 14:41:13 GMT

@nicobuko You actually do have a password, you created the password when you signed up for CE. The newer browser login experience uses your userid and an e-mailed OTP when you are using a recognized machine, but your password still exists. Try logging in using incognito on your browser, you should be prompted for your password. You'll also be able to change it this time. That's the password you'll use in your code (recommend an environment variable or make sure not to commit to a repo). Either of the methods in the documentation @Walter_C linked to should work once you get your password reset.

Re: MLflow: Connect python with Community Edition

nicobuko — Fri, 13 Dec 2024 14:24:14 GMT

@Rjdudley Thank you for your response. I do definetely not have a password. I have now created 3 or 4 accounts and I was never asked to create a password. Also in incognito mode I am not requested to enter a password. It is always using one time password (at signup and also at logging in and also in incognito mode).

Re: MLflow: Connect python with Community Edition

Rjdudley — Fri, 13 Dec 2024 15:08:53 GMT

@nicobuko I see what you mean, I just registered for CE using a different email address and I was not prompted for a password, it's OTP all the way. That's a change in the last few months, I apologize I didn't realize how much that signup changed.

@Advika_ @Walter_C The new signup process for CE does not create a password. The documentation you posted is now out of date and needs to be reviewed by your team, as does how mlflow can work without a password or PAT. This is a broken feature. If people are using mlflow in CE, I assume they signed up before the switch to OTPs and have a password (like I do).

Re: MLflow: Connect python with Community Edition

nicobuko — Fri, 13 Dec 2024 16:57:52 GMT

@Advika_ @Walter_C @Rjdudley It would be great if a fix could be made so that the feature works again in the future. Is there any way to stay up to date on when and how the feature will work again in the future?

Re: MLflow: Connect python with Community Edition

cwij95 — Fri, 20 Dec 2024 16:09:57 GMT

@nicobuko @Rjdudley I was using mlflow with Databricks CE previously, where I could use Basic Authentication details (username/ password) for login. I created that account 3 months ago. Now, since Basic Auth is no longer supported, and we cannot create any PAT for login, seems mlflow feature is no longer working with Databricks CE. I have added more details in this thread as well.

@Walter_C @Advika_ Appreciate if you could provide a fix to this issue, so that we can use mlflow with Databricks CE.

Re: MLflow: Connect python with Community Edition

Walter_C — Fri, 20 Dec 2024 16:11:54 GMT

I am currently looking with our internal teams if this will be provided in the near future, still waiting for confirmation.