Connect databricks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ03-31-2024 07:57 AM
I discovered recently mlflow managed by Databricks so I'm very new to this and I need some help.
Can someone explain for me clearly the steps to do to be able to track my runs into the Databricks API.
Here are the steps I followed :
1/ Installing Databricks CLI
2/ I sewed up authentication between Databricks CLI and my Databricks workspaces according to instructions here text
I checked the file cat ~/.databrickscfg and everything is fine
3/ I'm using Pycharm and I'm writing a python script including mlflow and I want to track the runs on my Databricks workspace Here is a part of my code :
mlflow.autolog( log_input_examples=True, log_model_signatures=True, log_models=True, disable=False, exclusive=False, disable_for_unsupported_versions=True, silent=False ) #mlflow.login() mlflow.set_tracking_uri("databricks") mlflow.set_experiment("mlflowAUS") with mlflow.start_run() as run: bestModel.fit(X_train, y_train) y_pred = bestModel.predict(X_test) runId = run.info.run_id mlflow.set_tag('mlflow.runName', datetime.datetime.now().strftime("%Y%m%d_%H%M%S")) mlflow.log_param("model_name", str(bestModel)[: str(bestModel).index("(")])
and if I add to my code
mlflow.login()
I got another time the same error but before the error I got : (I hided my the link with xxxx )
2024/03/30 01:40:44 INFO mlflow.utils.credentials: Successfully connected to MLflow hosted tracking server! Host: https://dbc-xxxxxx-xxxx.cloud.databricks.com.
What I don't understand why it says that I'm successfully connected after adding mlflow.login() but I still got the same error and I can't track my runs ? Please help me and I thank you in advance for the support
this code return error :
Traceback (most recent call last): File "/Users/kevin/PycharmProjects/Projects2024/nov23_continu_mlops_meteo/src/models/bestModel.py", line 184, in <module> mlflow.set_experiment("mlflowAUS") File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/tracking/fluent.py", line 142, in set_experiment experiment = client.get_experiment_by_name(experiment_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/tracking/client.py", line 539, in get_experiment_by_name return self._tracking_client.get_experiment_by_name(name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/tracking/_tracking_service/client.py", line 236, in get_experiment_by_name return self.store.get_experiment_by_name(name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/store/tracking/rest_store.py", line 323, in get_experiment_by_name response_proto = self._call_endpoint(GetExperimentByName, req_body) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/store/tracking/rest_store.py", line 60, in _call_endpoint return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/utils/rest_utils.py", line 220, in call_endpoint response = verify_rest_response(response, endpoint) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kevin/opt/anaconda3/envs/env_2023/lib/python3.11/site-packages/mlflow/utils/rest_utils.py", line 170, in verify_rest_response raise MlflowException(f"{base_msg}. Response body: '{response.text}'") mlflow.exceptions.MlflowException: API request to endpoint was successful but the response body was not in a valid JSON format. Response body: '<!doctype html>
You can see that if I use mlflow ui everything works without any issues

