I discovered recently mlflow managed by Databricks so I'm very new to this and I need some help.
Can someone explain for me clearly the steps to do to be able to track my runs into the Databricks API.
Here are the steps I followed :
1/ Installing Databricks CLI
2/ I sewed up authentication between Databricks CLI and my Databricks workspaces according to instructions here text
I checked the file cat ~/.databrickscfg and everything is fine
mlflow.autolog(
log_input_examples=True,
log_model_signatures=True,
log_models=True,
disable=False,
exclusive=False,
disable_for_unsupported_versions=True,
silent=False
)
#mlflow.login()
mlflow.set_tracking_uri("databricks")
mlflow.set_experiment("mlflowAUS")
with mlflow.start_run() as run:
bestModel.fit(X_train, y_train)
y_pred = bestModel.predict(X_test)
runId = run.info.run_id
mlflow.set_tag('mlflow.runName', datetime.datetime.now().strftime("%Y%m%d_%H%M%S"))
mlflow.log_param("model_name", str(bestModel)[: str(bestModel).index("(")])and if I add to my code
mlflow.login()
I got another time the same error but before the error I got : (I hided my the link with xxxx )
2024/03/30 01:40:44 INFO mlflow.utils.credentials: Successfully connected to MLflow hosted tracking server! Host: https://dbc-xxxxxx-xxxx.cloud.databricks.com.
What I don't understand why it says that I'm successfully connected after adding mlflow.login() but I still got the same error and I can't track my runs ? Please help me and I thank you in advance for the support
this code return error :
Traceback (most recent call last):
File "/Users/kevin/PycharmProjects/Projects2024/nov23_continu_mlops_meteo/src/models/bestModel.py", line 184, in <module>
mlflow.set_experiment("mlflowAUS")
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/tracking/fluent.py", line 142, in set_experiment
experiment = client.get_experiment_by_name(experiment_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/tracking/client.py", line 539, in get_experiment_by_name
return self._tracking_client.get_experiment_by_name(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/tracking/_tracking_service/client.py", line 236, in get_experiment_by_name
return self.store.get_experiment_by_name(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/store/tracking/rest_store.py", line 323, in get_experiment_by_name
response_proto = self._call_endpoint(GetExperimentByName, req_body)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/store/tracking/rest_store.py", line 60, in _call_endpoint
return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/utils/rest_utils.py", line 220, in call_endpoint
response = verify_rest_response(response, endpoint)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/utils/rest_utils.py", line 170, in verify_rest_response
raise MlflowException(f"{base_msg}. Response body: '{response.text}'")
mlflow.exceptions.MlflowException: API request to endpoint was successful but the response body was not in a valid JSON format. Response body: '<!doctype html>You can see that if I use mlflow ui everything works without any issues
