kdatt
Databricks Partner

‎10-19-2024 03:26 PM

I had same issue while trying to call notebook from workflow. I was able to do what you did. But it needs new experiment name for each run, so I had to do this:

# Set the experiment
experiment_name = f"/Workspace/MLOps/{env}/experiment/{experiment}_{time.strftime('%Y-%m-%d_%H-%M-%S')}"
mlflow.set_experiment(experiment_name)
 
But this assigns a new experiment ID each run which doesnt work for me as I was hardcoding that ID for inference.
Not sure whats the best option here.
0 Kudos