- 4025 Views
- 3 replies
- 0 kudos
Error installing datasets needed for LLM course
I signed up for this course via Databricks Academy : LLMs: Application through Production However I am getting this error when trying to download the needed datasets for the course:Installing datasets:| from "wasbs://courseware@dbacademy.blob.core.wi...
- 4025 Views
- 3 replies
- 0 kudos
- 0 kudos
You would need to install the python library. You can either:1) Run %pip install datasets2) Put it as part of the PyPi packages to load in your cluster This should solve your issue
- 0 kudos
- 1782 Views
- 2 replies
- 0 kudos
AutoML split with dt column not working properly
I am using AutoML and want to split my data to train/validation and test using a dt column (one date for train one different date for validation and a third date for test. The problem that the autoML fails, there are only training metrics (no valiat...
- 1782 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello! Did you try specify a column name as manual split column? Then you can fully control which rows are in train / validate / test splits: https://docs.databricks.com/en/machine-learning/automl/automl-data-preparation.html#split-data-for-regressi...
- 0 kudos
- 3005 Views
- 3 replies
- 0 kudos
Error using score_batch for batch inference
Hey everybody,I have been learning to use the Databricks feature store and I was trying to train the model using the stored features and compute batch inference. I am getting an error though, running prediction using score_batch, I have been getting ...
- 3005 Views
- 3 replies
- 0 kudos
- 0 kudos
Hey @Kumaran, I am using a Random forest classifier however I have tried to set the max depth to none since it is the default value but the error still exists.
- 0 kudos
- 1006 Views
- 0 replies
- 0 kudos
ModuleNotFoundError: No module named 'model_train' when using mlflow.sklearn.load_model
Hello,I have multiple versions of a model registered in model registry. When I am trying to load any other version except model version 1 by mlflow.sklearn.load_model(f"models:/{model_name}/{model_version}")I am getting ModuleNotFoundError: No module...
- 1006 Views
- 0 replies
- 0 kudos
- 709 Views
- 0 replies
- 0 kudos
Serving Endpoint Container Image Creation Fails
Hello, yesterday I send this message but I guess some AI flagging tool or non-technical moderator thought error logs are spam so no one could see my message. Thus, I am restating my problem without error logs this time.Essentially, after I train my m...
- 709 Views
- 0 replies
- 0 kudos
- 1658 Views
- 2 replies
- 0 kudos
TrainingSet schema difference during training and inference
Hi,I'm using the Feature Store to train an ml model and log it using MLflow and FeatureStoreClient(). This model is then used for inference.I understand the schema of the TrainingSet should not differ between training time and inference time. However...
- 1658 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Quinten,You can consider creating a custom feature group to store the "weight" column during training. This way, you can keep the schema of the TrainingSet consistent between training and inference time.Here are the steps you can follow:Create a...
- 0 kudos
- 1683 Views
- 2 replies
- 0 kudos
FeatureEngineeringClient failing to run inference with mlflow.spark flavor
I am using Databricks FeatureEngineeringClient to log my spark.ml model for batch inference. I use the ALS model on the movielens dataset. My dataset has three columns: user_id, item_id and rankhere is my code to prepare the dataset:fe_data = fe.crea...
- 1683 Views
- 2 replies
- 0 kudos
- 0 kudos
@KumaranT I did it already with the same result import mlflow.pyfunc # Load the model as a PyFuncModel model = mlflow.pyfunc.load_model(model_uri=f"{model_version_uri}") # Create a Spark UDF for scoring predict_udf = mlflow.pyfunc.spark_udf(spark, ...
- 0 kudos
- 1808 Views
- 1 replies
- 0 kudos
Issues with experiment errors over the last two weeks
Hi,I use Azure Databricks in the North Central US region and have had some issues over the last two weeks. Three weeks ago, I was able to run a forecast experiment. Last week I got this error on 7/24:[UNRESOLVED_COLUMN.WITH_SUGGESTION] A column, va...
- 1808 Views
- 1 replies
- 0 kudos
- 3078 Views
- 1 replies
- 0 kudos
Received Fatal error: The Python kernel is unresponsive.
I am running a databricks job on a cluster and I keep running into the following issue (pasted below in bold) The job trains a machine learning model on a modestly sized dataset (~ half GB). Note that I use pandas dataframes for the data, sklearn for...
- 3078 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @HappyScientist,Can you increase the memory size of your cluster and try again?
- 0 kudos
- 1548 Views
- 1 replies
- 0 kudos
AutoML workflows will no longer run with job compute
We have a few workflows that have been running fine with job compute (runtime 14x). They started failing on 6/3 with the following error: The cluster [xxx] is not an all-purpose cluster. existing_cluster_id only supports all-purpose cluster IDs. I w...
- 1548 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @c3,We can see this Automl issue got fixed, can you check whether you are getting the same issue?
- 0 kudos
- 1911 Views
- 1 replies
- 0 kudos
Error - Langchain to interact with a SQL database
I am using databricks community edition to use langchain on SQL database in databricks.I am following this link: Interact with SQL database - DatabricksBut I am facing issue on this line: db = SQLDatabase.from_databricks(catalog="samples", schema="ny...
- 1911 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @abd,Can you check upgrading the SQL driver?
- 0 kudos
- 2758 Views
- 1 replies
- 0 kudos
MLflow autolging is not registering my experiments
When training a any ML model in a Databricks notebook, after calling model.fit() and train the model, before the model was automatically saved, but now is giving me this error:WARNING mlflow.utils.autologging_utils: Encountered unexpected error durin...
- 2758 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @espartaco,The error message shows that there's an issue with SSL certificate verification when trying to connect to the Azure storage endpointCheck network and firewall configurations: You need to ensure that the network and firewall configuratio...
- 0 kudos
- 1217 Views
- 2 replies
- 0 kudos
Applyinpandas executed twice
Hi,I have a dataframe containing records (sales) over time for +- 1000 different items, so based on these records each item has its own timeseries. The goal is to make predictions for each of these items. Since the behaviour of these items is very di...
- 1217 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @fh ,To avoid this double execution, you can try using the concurrent.futures module in Python to parallelize the training of your models. This module provides a high-level interface for asynchronously executing callables.
- 0 kudos
- 1011 Views
- 2 replies
- 0 kudos
Databricks documentation for training a local LLM
Im in the process of training a chat-bot for my team to use to learn about databricks and relevant tools quickly. Is there a place that I can easily (and legally) grab learning material in PDF or text?
- 1011 Views
- 2 replies
- 0 kudos
- 721 Views
- 1 replies
- 0 kudos
error tu run btyd model
I run the model in april and ok but today I need run the model and I have error and it is not possible continue I change the penalizer_coef and nothing # fit a model with a larger penalizer coefficientbgf_engagement = BetaGeoFitter(penalizer_coef=100...
- 721 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @chagoo,To fix this, try lowering the penalizer coefficient, checking the data quality for anomalies, scaling the data, increasing the number of iterations, or experimenting with different initial parameters. These steps should help resolve the co...
- 0 kudos
