- 1461 Views
- 2 replies
- 0 kudos
Applyinpandas executed twice
Hi,I have a dataframe containing records (sales) over time for +- 1000 different items, so based on these records each item has its own timeseries. The goal is to make predictions for each of these items. Since the behaviour of these items is very di...
- 1461 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @fh ,To avoid this double execution, you can try using the concurrent.futures module in Python to parallelize the training of your models. This module provides a high-level interface for asynchronously executing callables.
- 0 kudos
- 1189 Views
- 2 replies
- 0 kudos
Databricks documentation for training a local LLM
Im in the process of training a chat-bot for my team to use to learn about databricks and relevant tools quickly. Is there a place that I can easily (and legally) grab learning material in PDF or text?
- 1189 Views
- 2 replies
- 0 kudos
- 866 Views
- 1 replies
- 0 kudos
error tu run btyd model
I run the model in april and ok but today I need run the model and I have error and it is not possible continue I change the penalizer_coef and nothing # fit a model with a larger penalizer coefficientbgf_engagement = BetaGeoFitter(penalizer_coef=100...
- 866 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @chagoo,To fix this, try lowering the penalizer coefficient, checking the data quality for anomalies, scaling the data, increasing the number of iterations, or experimenting with different initial parameters. These steps should help resolve the co...
- 0 kudos
- 1647 Views
- 1 replies
- 0 kudos
Debugging using vscode & databricks connect
Hi allI'm facing some difficulties when I use DataBricks Connect to debug my ML solution. A long story short, I want to investigate a few variables after I've conducted training. With the debugger at hand, I can simply place a breakpoint on the line ...
- 1647 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @EijayK, Ensure that the package is installed on the cluster itself, which you can verify through the cluster's library installation logs. Additionally, make sure your cluster meets all Databricks Connect requirements, including proper configurati...
- 0 kudos
- 1394 Views
- 2 replies
- 2 kudos
Feature Store - lookback_window does not work with primary keys of "date" type
I just discovered what I believe is a bug in Feature Store. The expected value (of the "value" column) is 'NULL' but the actual value is "a". If I instead change the format to timestamp of the "date" column (i.e. removes the .date() in the generation...
- 1394 Views
- 2 replies
- 2 kudos
- 2 kudos
Thank you for answering. Yes, that is also what I figured out. In other words the lookback_window argument only works when using timestamp format for the primary key. I cannot see that this behavior is described in the documentation.
- 2 kudos
- 7542 Views
- 2 replies
- 2 kudos
Resolved! How to search the run id of an experiment run created in another notebook?
Hello,I have created an experiment using with mlflow.start_run(run_name='experment_1'):in a notebook say 'notebook_1'. In the 'Experiments' tab if I click on 'notebook_1', I am able to see 'experiment_1'. Now I am trying to search the experiment in ...
- 7542 Views
- 2 replies
- 2 kudos
- 1510 Views
- 0 replies
- 0 kudos
large scale yolo inference
I have 50 Million Images sitting on s3 I have a Yolov8 model trained with ultralytics and want to run inference on those images. I suspect I should be running inference using ML flow, but I am confused on how. I don't need to track experiments/traini...
- 1510 Views
- 0 replies
- 0 kudos
- 7474 Views
- 4 replies
- 1 kudos
Vector Search Index Sync fails in Initializing
Vector Search Index Sync fails in Initializing. This index table was already up and running, and when I tried to sync it, it failed in Initializing. See the attached.
- 7474 Views
- 4 replies
- 1 kudos
- 1 kudos
The issue for us was most likely that we used CPU compute for the deployed embedding model, switching to GPU (small) solved the issue.
- 1 kudos
- 5892 Views
- 3 replies
- 0 kudos
Resolved! Initializing Vector Search index Sync failes with Failed to resolve flow: '__online_index_view'
When setting up a vector search in databricks using the bge_m3 (Version 1) embedding model available in system.ai schema, the setup runs for 20 minutes or so and then fails. Querying the served embedding models from the browser works perfectly fine. ...
- 5892 Views
- 3 replies
- 0 kudos
- 0 kudos
The issue was most likely to use a CPU compute for the deployed model, switching to GPU (small) solved the issue.
- 0 kudos
- 7041 Views
- 8 replies
- 0 kudos
Feature Store Model Serving endpoint
Hi,I am trying to deploy my model which was logged by featureStoreEngineering client as a serving endpoint in Databricks. But I am facing following error: The Databricks Lookup client from databricks-feature-lookup and Databricks Feature Store clie...
- 7041 Views
- 8 replies
- 0 kudos
- 0 kudos
Hi @damselfly20 unfortunately I can't help much with that as I've never worked with RAGs. Are you sure it's the same error though? @NaeemS's and my errors seems to be Java related and yours MLflow related.
- 0 kudos
- 2567 Views
- 1 replies
- 1 kudos
Resolved! Vectorsearch ConnectionResetError Max retries exceeded
Hi,we are serving a unity catalog langchain model with databricks model serving. When I run the predict() function on the model in a notebook, I get the expected output. But when I query the served model, errors occur in the service logs:Error messag...
- 2567 Views
- 1 replies
- 1 kudos
- 1 kudos
downgrading langchain-community to version 0.2.4 solved my problem.
- 1 kudos
- 8493 Views
- 4 replies
- 3 kudos
Passing parameters in Databricks workflows
Hi Databricks, we have created several Databricks workflows and the `json-definition.json` for the same is stored inside version control i.e. GitHub. There are several parameters which are referred from params.json inside this job definition but the ...
- 8493 Views
- 4 replies
- 3 kudos
- 3 kudos
Have you considered using Databricks Asset Bundles? Very easy to parameterize!
- 3 kudos
- 5000 Views
- 4 replies
- 1 kudos
Resolved! Model flavour using feature store model training log_model()
Hi I'm have succesfully registered my model using the feature engineering client with the following codes:with mlflow.start_run(): # Calculate the ratio of negative class samples to positive class samples ratio = (len(y_train) - y_train.sum()...
- 5000 Views
- 4 replies
- 1 kudos
- 1 kudos
Thanks for your reply @robbe - yes I have created a custom pyfunc model which I can now use fe.score_batch() to return probabilities. Here is the code:# Calculate the ratio of negative class samples to positive class samples ratio = (len(y_train) - y...
- 1 kudos
- 5589 Views
- 2 replies
- 0 kudos
Can't load model from UC due to DBFS issue
I want to load a model I have registered in Unity Catalog using a Shared cluster, but it seems to be trying to use dbfs under the hood and it gives me an error.I am using DBR 13.3 LTS and mlflow-skinny[databricks]==2.14.3My code import mlflow mlflow...
- 5589 Views
- 2 replies
- 0 kudos
- 0 kudos
Have you tried to tell MLFlow to look for models in UC? mlflow.set_registry_uri("databricks-uc") Edit: never mind I see you have already. It shouldn't do/search for anything on DBFS anymore when setting this option so it is a bit strange. Shared clus...
- 0 kudos
- 1066 Views
- 0 replies
- 0 kudos
Creating an Input Schema for Multiple DataFrames in MLflow
Hi everyone,I am working with MLflow version 2.5.0 and need to create an input_schema for my model. My data schema is divided into several DataFrames, for example:{"dataframe_split": { "columns": ["ClientGuid", "Instance", "TypeScore", ...], ...
- 1066 Views
- 0 replies
- 0 kudos
