Hello,I have multiple versions of a model registered in model registry. When I am trying to load any other version except model version 1 by mlflow.sklearn.load_model(f"models:/{model_name}/{model_version}")I am getting ModuleNotFoundError: No module...
Hello, yesterday I send this message but I guess some AI flagging tool or non-technical moderator thought error logs are spam so no one could see my message. Thus, I am restating my problem without error logs this time.Essentially, after I train my m...
Hi,I'm using the Feature Store to train an ml model and log it using MLflow and FeatureStoreClient(). This model is then used for inference.I understand the schema of the TrainingSet should not differ between training time and inference time. However...
Hi @Quinten,You can consider creating a custom feature group to store the "weight" column during training. This way, you can keep the schema of the TrainingSet consistent between training and inference time.Here are the steps you can follow:Create a...
I am using Databricks FeatureEngineeringClient to log my spark.ml model for batch inference. I use the ALS model on the movielens dataset. My dataset has three columns: user_id, item_id and rankhere is my code to prepare the dataset:fe_data = fe.crea...
@KumaranT I did it already with the same result import mlflow.pyfunc # Load the model as a PyFuncModel model = mlflow.pyfunc.load_model(model_uri=f"{model_version_uri}") # Create a Spark UDF for scoring predict_udf = mlflow.pyfunc.spark_udf(spark, ...
Hi,I use Azure Databricks in the North Central US region and have had some issues over the last two weeks. Three weeks ago, I was able to run a forecast experiment. Last week I got this error on 7/24:[UNRESOLVED_COLUMN.WITH_SUGGESTION] A column, va...
I am running a databricks job on a cluster and I keep running into the following issue (pasted below in bold) The job trains a machine learning model on a modestly sized dataset (~ half GB). Note that I use pandas dataframes for the data, sklearn for...
Hi @HappyScientist,Can you increase the memory size of your cluster and try again?
We have a few workflows that have been running fine with job compute (runtime 14x). They started failing on 6/3 with the following error: The cluster [xxx] is not an all-purpose cluster. existing_cluster_id only supports all-purpose cluster IDs. I w...
Hi @c3,We can see this Automl issue got fixed, can you check whether you are getting the same issue?
I am using databricks community edition to use langchain on SQL database in databricks.I am following this link: Interact with SQL database - DatabricksBut I am facing issue on this line: db = SQLDatabase.from_databricks(catalog="samples", schema="ny...
Hi @abd,Can you check upgrading the SQL driver?
When training a any ML model in a Databricks notebook, after calling model.fit() and train the model, before the model was automatically saved, but now is giving me this error:WARNING mlflow.utils.autologging_utils: Encountered unexpected error durin...
Hi @espartaco,The error message shows that there's an issue with SSL certificate verification when trying to connect to the Azure storage endpointCheck network and firewall configurations: You need to ensure that the network and firewall configuratio...
Hi,I have a dataframe containing records (sales) over time for +- 1000 different items, so based on these records each item has its own timeseries. The goal is to make predictions for each of these items. Since the behaviour of these items is very di...
Hi @fh ,To avoid this double execution, you can try using the concurrent.futures module in Python to parallelize the training of your models. This module provides a high-level interface for asynchronously executing callables.
Im in the process of training a chat-bot for my team to use to learn about databricks and relevant tools quickly. Is there a place that I can easily (and legally) grab learning material in PDF or text?
I run the model in april and ok but today I need run the model and I have error and it is not possible continue I change the penalizer_coef and nothing # fit a model with a larger penalizer coefficientbgf_engagement = BetaGeoFitter(penalizer_coef=100...
Hi @chagoo,To fix this, try lowering the penalizer coefficient, checking the data quality for anomalies, scaling the data, increasing the number of iterations, or experimenting with different initial parameters. These steps should help resolve the co...
Hi allI'm facing some difficulties when I use DataBricks Connect to debug my ML solution. A long story short, I want to investigate a few variables after I've conducted training. With the debugger at hand, I can simply place a breakpoint on the line ...
Hi @EijayK, Ensure that the package is installed on the cluster itself, which you can verify through the cluster's library installation logs. Additionally, make sure your cluster meets all Databricks Connect requirements, including proper configurati...
I just discovered what I believe is a bug in Feature Store. The expected value (of the "value" column) is 'NULL' but the actual value is "a". If I instead change the format to timestamp of the "date" column (i.e. removes the .date() in the generation...
Thank you for answering. Yes, that is also what I figured out. In other words the lookback_window argument only works when using timestamp format for the primary key. I cannot see that this behavior is described in the documentation.
Hello,I have created an experiment using with mlflow.start_run(run_name='experment_1'):in a notebook say 'notebook_1'. In the 'Experiments' tab if I click on 'notebook_1', I am able to see 'experiment_1'. Now I am trying to search the experiment in ...
