cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Cannot set experiment in a non-ML runtime

yopbibo
Contributor II

Hello,

If we:

%pip install mlflow
import mlflow
mlflow.set_experiment(experiment_name = '/Shared/xx')

we get:

InvalidConfigurationError: You haven't configured the CLI yet! Please configure by entering `/databricks/python_shell/scripts/db_ipykernel_launcher.py configure`

Possible to get it working in recent non-ML runtime?
Thanks,

3 REPLIES 3

Kaniz_Fatma
Community Manager
Community Manager

Hi @yopbibo ,

Yes, it is possible to use the Python mlflow package in Databricks without running into the InvalidConfigurationError you've encountered. The error message suggests that the MLflow CLI needs to be configured before using the mlflow package.

To avoid this error, you can avoid using the mlflow CLI and use MLflow tracking via its API. Here is an example of how to log an experiment run in MLflow without the CLI configuration:

import mlflow

# Set the experiment using the experiment ID
mlflow.set_experiment('/Shared/xx')

# Start a new run
with mlflow.start_run() as run:
    # Log parameters
    mlflow.log_param('learning_rate', 0.01)
    mlflow.log_param('max_depth', 5)

    # Log metrics
    mlflow.log_metric('train_loss', 0.5)
    mlflow.log_metric('valid_loss', 1.0)

    # Log artifacts
    filename = 'model.pkl'
    # ... train model and create model artifact ...

    with open(filename, 'rb') as f:
        mlflow.log_artifact(f, artifact_path=filename)

The `mlflow` package provides several API functions to log data such as metrics, parameters, and artifacts to an experiment run.

To avoid running into `InvalidConfigurationError`, make sure that you have installed the `mlflow` package in your Databricks environment using `%pip install mlflow`.

Once you have configured your experiment run with `mlflow`, you can use the `log_param`, `log_metric`, and `log_artifact` functions to log data to the run. Note that the `log_param` and `log_metric` functions require a name and a value, while the `log_artifact` function requires a file object and an artifact path.

By using the `mlflow` API in your code, you can avoid the need to configure the CLI and directly interact with your experiment runs in MLflow.

 

yopbibo
Contributor II

hi,
I wrote above that I: %pip install mlflow, then mlflow.set_experiment() and that fails.
You advice me to do: %pip install mlflow, mlflow.set_experiment()
That will fail. ๐Ÿ˜ž

yopbibo
Contributor II

%pip install mlflow
dbutils.library.restartPython()

that works

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group