MLflow: Connect python with Community Edition

[nicobuko]

Hello,

I am new to databricks and want to work with MLFlow in the Databricks Community Edition. In python i am using mlflow.login(). This requests me to enter a password. But i do not have any password due to the fact that databricks login only required a one time password.

Is there another way to connect python Code to the community Edition? Or how do i create a password?

In the settings in the community Edition there is a password changing functionality but this one Requires to enter the old password which obviously does not exist.

It would be great if someone could Provide some help! Thank you very much!

[Walter_C]

You can try the following:

Instead of using mlflow.login(), you can set the tracking URI and the token directly in your code. Here is an example:

import mlflow

# Set the tracking URI to your Databricks workspace
mlflow.set_tracking_uri("https://community.cloud.databricks.com")

# Set the Databricks token
token = "YOUR_PERSONAL_ACCESS_TOKEN"

# Configure the MLFlow client to use the token
mlflow.set_experiment("your-experiment-name")
mlflow.start_run()
mlflow.log_param("param1", 5)
mlflow.log_metric("metric1", 0.85)
mlflow.end_run()

[nicobuko]

@Walter_C Thank you for your fast answer 🙂

Where can I generate such a token in the community edition? I did not see an option to generate a token when I have looked into the community Edition.

[Walter_C]

Under Settings > Developer it does not allow you to?

[Rjdudley]

The Community Edition is very feature limited.  PATs are one feature which is not available.  None of the SQL Warehouse features are available, nor are most of the AI/ML features.

[Walter_C]

Got it, as per our docs it is suggested the creation of a token for Community edition as you can see here https://docs.databricks.com/en/mlflow/access-hosted-tracking-server.html#step-1-configure-your-envir... it states that for Community Edition you can use any of the two approaches listed there, can you go through the doc and let me know if this are the methods you have attempted?

[nicobuko]

@Rjdudley Thank you for your answer but I am wondering about your answer because the mlflow documentation explicitly says that the CE could be used: https://mlflow.org/docs/latest/getting-started/running-notebooks/index.html

But the way described in the docs via mlflow.login is obviously not functional because i do not have a password.

@Walter_C Before your comment i had seen the doc that you have linked once. But both Suggestions there are relying on the need to have a password which a User obviously does not have because of one time passwords. So the doc seems to be outdated and for me there seems to be currently no way to connect...

[Rjdudley]

@nicobuko You actually do have a password, you created the password when you signed up for CE.  The newer browser login experience uses your userid and an e-mailed OTP when you are using a recognized machine, but your password still exists.  Try logging in using incognito on your browser, you should be prompted for your password.  You'll also be able to change it this time.  That's the password you'll use in your code (recommend an environment variable or make sure not to commit to a repo).  Either of the methods in the documentation @Walter_C linked to should work once you get your password reset.

[Advika]

Hello @nicobuko!

You can find helpful resources for Databricks Community Edition here. If the available resource doesn’t resolve your concern, feel free to submit a ticket with Databricks Support team for further assistance.

Thank you.

[nicobuko]

@Rjdudley Thank you for your response. I do definetely not have a password. I have now created 3 or 4 accounts and I was never asked to create a password. Also in incognito mode I am not requested to enter a password. It is always using one time password (at signup and also at logging in and also in incognito mode).

[Rjdudley]

@nicobuko I see what you mean, I just registered for CE using a different email address and I was not prompted for a password, it's OTP all the way.  That's a change in the last few months, I apologize I didn't realize how much that signup changed.

@Advika @Walter_C The new signup process for CE does not create a password.  The documentation you posted is now out of date and needs to be reviewed by your team, as does how mlflow can work without a password or PAT.  This is a broken feature.  If people are using mlflow in CE, I assume they signed up before the switch to OTPs and have a password (like I do).

[nicobuko]

@Advika @Walter_C @Rjdudley It would be great if a fix could be made so that the feature works again in the future. Is there any way to stay up to date on when and how the feature will work again in the future?

[cwij95]

@nicobuko @Rjdudley I was using mlflow with Databricks CE previously, where I could use Basic Authentication details (username/ password) for login. I created that account 3 months ago. Now, since Basic Auth is no longer supported, and we cannot create any PAT for login, seems mlflow feature is no longer working with Databricks CE. I have added more details in this thread as well.

@Walter_C @Advika Appreciate if you could provide a fix to this issue, so that we can use mlflow with Databricks CE.

[Walter_C]

I am currently looking with our internal teams if this will be provided in the near future, still waiting for confirmation.