Hi!I was wondering if any of you has ever dealt with Feature tables and null values (more specifically, via feature engineering objects, rather than feature store, although I don't think it really matters).In brief, null values are allowed to be stor...
Hey everybody,I have been learning to use the Databricks feature store and I was trying to train the model using the stored features and compute batch inference. I am getting an error though, running prediction using score_batch, I have been getting ...
Hey @Kumaran, I am using a Random forest classifier however I have tried to set the max depth to none since it is the default value but the error still exists.
Hello,I have multiple versions of a model registered in model registry. When I am trying to load any other version except model version 1 by mlflow.sklearn.load_model(f"models:/{model_name}/{model_version}")I am getting ModuleNotFoundError: No module...
Hello, yesterday I send this message but I guess some AI flagging tool or non-technical moderator thought error logs are spam so no one could see my message. Thus, I am restating my problem without error logs this time.Essentially, after I train my m...
I use DAB for deploying Jobs, I want to add a specific Task in dev only but not in staging or prod. Is there any way to achieve this using DAB ?
Hi,I'm using the Feature Store to train an ml model and log it using MLflow and FeatureStoreClient(). This model is then used for inference.I understand the schema of the TrainingSet should not differ between training time and inference time. However...
Hi @Quinten,You can consider creating a custom feature group to store the "weight" column during training. This way, you can keep the schema of the TrainingSet consistent between training and inference time.Here are the steps you can follow:Create a...
I am using Databricks FeatureEngineeringClient to log my spark.ml model for batch inference. I use the ALS model on the movielens dataset. My dataset has three columns: user_id, item_id and rankhere is my code to prepare the dataset:fe_data = fe.crea...
@KumaranT I did it already with the same result import mlflow.pyfunc # Load the model as a PyFuncModel model = mlflow.pyfunc.load_model(model_uri=f"{model_version_uri}") # Create a Spark UDF for scoring predict_udf = mlflow.pyfunc.spark_udf(spark, ...
Hi,I use Azure Databricks in the North Central US region and have had some issues over the last two weeks. Three weeks ago, I was able to run a forecast experiment. Last week I got this error on 7/24:[UNRESOLVED_COLUMN.WITH_SUGGESTION] A column, va...
Hi @SamGreene, This error indicates that the SQL engine cannot find the column ds. This could be due to changes in your data schema or the way the query is constructed. Ensure that the column ds exists in your dataset and is correctly referenced in ...
I am running a databricks job on a cluster and I keep running into the following issue (pasted below in bold) The job trains a machine learning model on a modestly sized dataset (~ half GB). Note that I use pandas dataframes for the data, sklearn for...
Hi @HappyScientist,Can you increase the memory size of your cluster and try again?
We have a few workflows that have been running fine with job compute (runtime 14x). They started failing on 6/3 with the following error: The cluster [xxx] is not an all-purpose cluster. existing_cluster_id only supports all-purpose cluster IDs. I w...
Hi @c3,We can see this Automl issue got fixed, can you check whether you are getting the same issue?
I am using databricks community edition to use langchain on SQL database in databricks.I am following this link: Interact with SQL database - DatabricksBut I am facing issue on this line: db = SQLDatabase.from_databricks(catalog="samples", schema="ny...
When training a any ML model in a Databricks notebook, after calling model.fit() and train the model, before the model was automatically saved, but now is giving me this error:WARNING mlflow.utils.autologging_utils: Encountered unexpected error durin...
Hi @espartaco,The error message shows that there's an issue with SSL certificate verification when trying to connect to the Azure storage endpointCheck network and firewall configurations: You need to ensure that the network and firewall configuratio...
Hi,I have a dataframe containing records (sales) over time for +- 1000 different items, so based on these records each item has its own timeseries. The goal is to make predictions for each of these items. Since the behaviour of these items is very di...
Hi @fh ,To avoid this double execution, you can try using the concurrent.futures module in Python to parallelize the training of your models. This module provides a high-level interface for asynchronously executing callables.
Im in the process of training a chat-bot for my team to use to learn about databricks and relevant tools quickly. Is there a place that I can easily (and legally) grab learning material in PDF or text?
I run the model in april and ok but today I need run the model and I have error and it is not possible continue I change the penalizer_coef and nothing # fit a model with a larger penalizer coefficientbgf_engagement = BetaGeoFitter(penalizer_coef=100...
Hi @chagoo,To fix this, try lowering the penalizer coefficient, checking the data quality for anomalies, scaling the data, increasing the number of iterations, or experimenting with different initial parameters. These steps should help resolve the co...
