- 837 Views
- 2 replies
- 2 kudos
Feature Store - lookback_window does not work with primary keys of "date" type
I just discovered what I believe is a bug in Feature Store. The expected value (of the "value" column) is 'NULL' but the actual value is "a". If I instead change the format to timestamp of the "date" column (i.e. removes the .date() in the generation...
- 837 Views
- 2 replies
- 2 kudos
- 2 kudos
Thank you for answering. Yes, that is also what I figured out. In other words the lookback_window argument only works when using timestamp format for the primary key. I cannot see that this behavior is described in the documentation.
- 2 kudos
- 5649 Views
- 2 replies
- 2 kudos
Resolved! How to search the run id of an experiment run created in another notebook?
Hello,I have created an experiment using with mlflow.start_run(run_name='experment_1'):in a notebook say 'notebook_1'. In the 'Experiments' tab if I click on 'notebook_1', I am able to see 'experiment_1'. Now I am trying to search the experiment in ...
- 5649 Views
- 2 replies
- 2 kudos
- 980 Views
- 0 replies
- 0 kudos
large scale yolo inference
I have 50 Million Images sitting on s3 I have a Yolov8 model trained with ultralytics and want to run inference on those images. I suspect I should be running inference using ML flow, but I am confused on how. I don't need to track experiments/traini...
- 980 Views
- 0 replies
- 0 kudos
- 6081 Views
- 4 replies
- 1 kudos
Vector Search Index Sync fails in Initializing
Vector Search Index Sync fails in Initializing. This index table was already up and running, and when I tried to sync it, it failed in Initializing. See the attached.
- 6081 Views
- 4 replies
- 1 kudos
- 1 kudos
The issue for us was most likely that we used CPU compute for the deployed embedding model, switching to GPU (small) solved the issue.
- 1 kudos
- 3685 Views
- 3 replies
- 0 kudos
Resolved! Initializing Vector Search index Sync failes with Failed to resolve flow: '__online_index_view'
When setting up a vector search in databricks using the bge_m3 (Version 1) embedding model available in system.ai schema, the setup runs for 20 minutes or so and then fails. Querying the served embedding models from the browser works perfectly fine. ...
- 3685 Views
- 3 replies
- 0 kudos
- 0 kudos
The issue was most likely to use a CPU compute for the deployed model, switching to GPU (small) solved the issue.
- 0 kudos
- 4918 Views
- 8 replies
- 0 kudos
Feature Store Model Serving endpoint
Hi,I am trying to deploy my model which was logged by featureStoreEngineering client as a serving endpoint in Databricks. But I am facing following error: The Databricks Lookup client from databricks-feature-lookup and Databricks Feature Store clie...
- 4918 Views
- 8 replies
- 0 kudos
- 0 kudos
Hi @damselfly20 unfortunately I can't help much with that as I've never worked with RAGs. Are you sure it's the same error though? @NaeemS's and my errors seems to be Java related and yours MLflow related.
- 0 kudos
- 1712 Views
- 1 replies
- 1 kudos
Resolved! Vectorsearch ConnectionResetError Max retries exceeded
Hi,we are serving a unity catalog langchain model with databricks model serving. When I run the predict() function on the model in a notebook, I get the expected output. But when I query the served model, errors occur in the service logs:Error messag...
- 1712 Views
- 1 replies
- 1 kudos
- 1 kudos
downgrading langchain-community to version 0.2.4 solved my problem.
- 1 kudos
- 6896 Views
- 4 replies
- 3 kudos
Passing parameters in Databricks workflows
Hi Databricks, we have created several Databricks workflows and the `json-definition.json` for the same is stored inside version control i.e. GitHub. There are several parameters which are referred from params.json inside this job definition but the ...
- 6896 Views
- 4 replies
- 3 kudos
- 3 kudos
Have you considered using Databricks Asset Bundles? Very easy to parameterize!
- 3 kudos
- 3341 Views
- 4 replies
- 1 kudos
Resolved! Model flavour using feature store model training log_model()
Hi I'm have succesfully registered my model using the feature engineering client with the following codes:with mlflow.start_run(): # Calculate the ratio of negative class samples to positive class samples ratio = (len(y_train) - y_train.sum()...
- 3341 Views
- 4 replies
- 1 kudos
- 1 kudos
Thanks for your reply @robbe - yes I have created a custom pyfunc model which I can now use fe.score_batch() to return probabilities. Here is the code:# Calculate the ratio of negative class samples to positive class samples ratio = (len(y_train) - y...
- 1 kudos
- 4407 Views
- 2 replies
- 0 kudos
Can't load model from UC due to DBFS issue
I want to load a model I have registered in Unity Catalog using a Shared cluster, but it seems to be trying to use dbfs under the hood and it gives me an error.I am using DBR 13.3 LTS and mlflow-skinny[databricks]==2.14.3My code import mlflow mlflow...
- 4407 Views
- 2 replies
- 0 kudos
- 0 kudos
Have you tried to tell MLFlow to look for models in UC? mlflow.set_registry_uri("databricks-uc") Edit: never mind I see you have already. It shouldn't do/search for anything on DBFS anymore when setting this option so it is a bit strange. Shared clus...
- 0 kudos
- 795 Views
- 0 replies
- 0 kudos
Creating an Input Schema for Multiple DataFrames in MLflow
Hi everyone,I am working with MLflow version 2.5.0 and need to create an input_schema for my model. My data schema is divided into several DataFrames, for example:{"dataframe_split": { "columns": ["ClientGuid", "Instance", "TypeScore", ...], ...
- 795 Views
- 0 replies
- 0 kudos
- 3277 Views
- 4 replies
- 1 kudos
cluster sharing between different notebooks
I have two structured streaming notebooks running continuously for anomaly detection. Both notebooks import the same python module to mount the Azure blob storage, but each has its own container. Each notebook runs well when it has its own cluster. ...
- 3277 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @johnp , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback ...
- 1 kudos
- 3200 Views
- 3 replies
- 1 kudos
Attribute based access control in Unity catalog
Can I start using Attribute based access control ? Is it available now?
- 3200 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi, I want to use Attributed-Based Access Control, but I cannot find the option to create rules in my catalog. Is it already available in public preview?
- 1 kudos
- 23738 Views
- 4 replies
- 0 kudos
Resolved! databricks-cli
Hello! I am trying to use the databricks asset bundles through the webui on a databricks compute cluster. However to use this I need the databricks-cli library. I tried to install it on a cluster like described in the documentation using the curl com...
- 23738 Views
- 4 replies
- 0 kudos
- 0 kudos
Thank you for your help! I read over the part of the runtime of your cluster which has to be 15.0 or more in the documentation you linked. I checked and my compute was still on a LTS 14.3 runtime version, which was the cause.
- 0 kudos
- 2069 Views
- 1 replies
- 0 kudos
Cannot log SparkML model to Unity Catalog due to missing output signature
I am training Spark ML model (concretely a SynapseML LightGBM ) in Databricks using mlflow and autologWhen I try to register my model in Unity catalog I get the following error: MlflowException: Model passed for registration contained a signature th...
- 2069 Views
- 1 replies
- 0 kudos
