cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Cannot set experiment in a non-ML runtime

yopbibo
Contributor II

Hello,

If we:

%pip install mlflow
import mlflow
mlflow.set_experiment(experiment_name = '/Shared/xx')

we get:

InvalidConfigurationError: You haven't configured the CLI yet! Please configure by entering `/databricks/python_shell/scripts/db_ipykernel_launcher.py configure`

Possible to get it working in recent non-ML runtime?
Thanks,

3 REPLIES 3

Kaniz_Fatma
Community Manager
Community Manager

Hi @yopbibo ,

Yes, it is possible to use the Python mlflow package in Databricks without running into the InvalidConfigurationError you've encountered. The error message suggests that the MLflow CLI needs to be configured before using the mlflow package.

To avoid this error, you can avoid using the mlflow CLI and use MLflow tracking via its API. Here is an example of how to log an experiment run in MLflow without the CLI configuration:

import mlflow

# Set the experiment using the experiment ID
mlflow.set_experiment('/Shared/xx')

# Start a new run
with mlflow.start_run() as run:
    # Log parameters
    mlflow.log_param('learning_rate', 0.01)
    mlflow.log_param('max_depth', 5)

    # Log metrics
    mlflow.log_metric('train_loss', 0.5)
    mlflow.log_metric('valid_loss', 1.0)

    # Log artifacts
    filename = 'model.pkl'
    # ... train model and create model artifact ...

    with open(filename, 'rb') as f:
        mlflow.log_artifact(f, artifact_path=filename)

The `mlflow` package provides several API functions to log data such as metrics, parameters, and artifacts to an experiment run.

To avoid running into `InvalidConfigurationError`, make sure that you have installed the `mlflow` package in your Databricks environment using `%pip install mlflow`.

Once you have configured your experiment run with `mlflow`, you can use the `log_param`, `log_metric`, and `log_artifact` functions to log data to the run. Note that the `log_param` and `log_metric` functions require a name and a value, while the `log_artifact` function requires a file object and an artifact path.

By using the `mlflow` API in your code, you can avoid the need to configure the CLI and directly interact with your experiment runs in MLflow.

 

yopbibo
Contributor II

hi,
I wrote above that I: %pip install mlflow, then mlflow.set_experiment() and that fails.
You advice me to do: %pip install mlflow, mlflow.set_experiment()
That will fail. 😞

yopbibo
Contributor II

%pip install mlflow
dbutils.library.restartPython()

that works

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!