Hi,I use Azure Databricks in the North Central US region and have had some issues over the last two weeks. Three weeks ago, I was able to run a forecast experiment. Last week I got this error on 7/24:[UNRESOLVED_COLUMN.WITH_SUGGESTION] A column, va...
I am running a databricks job on a cluster and I keep running into the following issue (pasted below in bold) The job trains a machine learning model on a modestly sized dataset (~ half GB). Note that I use pandas dataframes for the data, sklearn for...
Hi @HappyScientist,Can you increase the memory size of your cluster and try again?
We have a few workflows that have been running fine with job compute (runtime 14x). They started failing on 6/3 with the following error: The cluster [xxx] is not an all-purpose cluster. existing_cluster_id only supports all-purpose cluster IDs. I w...
Hi @c3,We can see this Automl issue got fixed, can you check whether you are getting the same issue?
I am using databricks community edition to use langchain on SQL database in databricks.I am following this link: Interact with SQL database - DatabricksBut I am facing issue on this line: db = SQLDatabase.from_databricks(catalog="samples", schema="ny...
Hi @abd,Can you check upgrading the SQL driver?
When training a any ML model in a Databricks notebook, after calling model.fit() and train the model, before the model was automatically saved, but now is giving me this error:WARNING mlflow.utils.autologging_utils: Encountered unexpected error durin...
Hi @espartaco,The error message shows that there's an issue with SSL certificate verification when trying to connect to the Azure storage endpointCheck network and firewall configurations: You need to ensure that the network and firewall configuratio...
Hi,I have a dataframe containing records (sales) over time for +- 1000 different items, so based on these records each item has its own timeseries. The goal is to make predictions for each of these items. Since the behaviour of these items is very di...
Hi @fh ,To avoid this double execution, you can try using the concurrent.futures module in Python to parallelize the training of your models. This module provides a high-level interface for asynchronously executing callables.
Im in the process of training a chat-bot for my team to use to learn about databricks and relevant tools quickly. Is there a place that I can easily (and legally) grab learning material in PDF or text?
I run the model in april and ok but today I need run the model and I have error and it is not possible continue I change the penalizer_coef and nothing # fit a model with a larger penalizer coefficientbgf_engagement = BetaGeoFitter(penalizer_coef=100...
Hi @chagoo,To fix this, try lowering the penalizer coefficient, checking the data quality for anomalies, scaling the data, increasing the number of iterations, or experimenting with different initial parameters. These steps should help resolve the co...
Hi allI'm facing some difficulties when I use DataBricks Connect to debug my ML solution. A long story short, I want to investigate a few variables after I've conducted training. With the debugger at hand, I can simply place a breakpoint on the line ...
Hi @EijayK, Ensure that the package is installed on the cluster itself, which you can verify through the cluster's library installation logs. Additionally, make sure your cluster meets all Databricks Connect requirements, including proper configurati...
I just discovered what I believe is a bug in Feature Store. The expected value (of the "value" column) is 'NULL' but the actual value is "a". If I instead change the format to timestamp of the "date" column (i.e. removes the .date() in the generation...
Thank you for answering. Yes, that is also what I figured out. In other words the lookback_window argument only works when using timestamp format for the primary key. I cannot see that this behavior is described in the documentation.
Hello,I have created an experiment using with mlflow.start_run(run_name='experment_1'):in a notebook say 'notebook_1'. In the 'Experiments' tab if I click on 'notebook_1', I am able to see 'experiment_1'. Now I am trying to search the experiment in ...
I have 50 Million Images sitting on s3 I have a Yolov8 model trained with ultralytics and want to run inference on those images. I suspect I should be running inference using ML flow, but I am confused on how. I don't need to track experiments/traini...
Vector Search Index Sync fails in Initializing. This index table was already up and running, and when I tried to sync it, it failed in Initializing. See the attached.
The issue for us was most likely that we used CPU compute for the deployed embedding model, switching to GPU (small) solved the issue.
When setting up a vector search in databricks using the bge_m3 (Version 1) embedding model available in system.ai schema, the setup runs for 20 minutes or so and then fails. Querying the served embedding models from the browser works perfectly fine. ...
The issue was most likely to use a CPU compute for the deployed model, switching to GPU (small) solved the issue.
Hi,I am trying to deploy my model which was logged by featureStoreEngineering client as a serving endpoint in Databricks. But I am facing following error: The Databricks Lookup client from databricks-feature-lookup and Databricks Feature Store clie...
Hi @damselfly20 unfortunately I can't help much with that as I've never worked with RAGs. Are you sure it's the same error though? @NaeemS's and my errors seems to be Java related and yours MLflow related.
