- 1950 Views
- 1 replies
- 0 kudos
Debugging using vscode & databricks connect
Hi allI'm facing some difficulties when I use DataBricks Connect to debug my ML solution. A long story short, I want to investigate a few variables after I've conducted training. With the debugger at hand, I can simply place a breakpoint on the line ...
- 1950 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @EijayK, Ensure that the package is installed on the cluster itself, which you can verify through the cluster's library installation logs. Additionally, make sure your cluster meets all Databricks Connect requirements, including proper configurati...
- 0 kudos
- 1796 Views
- 2 replies
- 2 kudos
Feature Store - lookback_window does not work with primary keys of "date" type
I just discovered what I believe is a bug in Feature Store. The expected value (of the "value" column) is 'NULL' but the actual value is "a". If I instead change the format to timestamp of the "date" column (i.e. removes the .date() in the generation...
- 1796 Views
- 2 replies
- 2 kudos
- 2 kudos
Thank you for answering. Yes, that is also what I figured out. In other words the lookback_window argument only works when using timestamp format for the primary key. I cannot see that this behavior is described in the documentation.
- 2 kudos
- 8808 Views
- 2 replies
- 2 kudos
Resolved! How to search the run id of an experiment run created in another notebook?
Hello,I have created an experiment using with mlflow.start_run(run_name='experment_1'):in a notebook say 'notebook_1'. In the 'Experiments' tab if I click on 'notebook_1', I am able to see 'experiment_1'. Now I am trying to search the experiment in ...
- 8808 Views
- 2 replies
- 2 kudos
- 1717 Views
- 0 replies
- 0 kudos
large scale yolo inference
I have 50 Million Images sitting on s3 I have a Yolov8 model trained with ultralytics and want to run inference on those images. I suspect I should be running inference using ML flow, but I am confused on how. I don't need to track experiments/traini...
- 1717 Views
- 0 replies
- 0 kudos
- 8228 Views
- 4 replies
- 1 kudos
Vector Search Index Sync fails in Initializing
Vector Search Index Sync fails in Initializing. This index table was already up and running, and when I tried to sync it, it failed in Initializing. See the attached.
- 8228 Views
- 4 replies
- 1 kudos
- 1 kudos
The issue for us was most likely that we used CPU compute for the deployed embedding model, switching to GPU (small) solved the issue.
- 1 kudos
- 7136 Views
- 3 replies
- 0 kudos
Resolved! Initializing Vector Search index Sync failes with Failed to resolve flow: '__online_index_view'
When setting up a vector search in databricks using the bge_m3 (Version 1) embedding model available in system.ai schema, the setup runs for 20 minutes or so and then fails. Querying the served embedding models from the browser works perfectly fine. ...
- 7136 Views
- 3 replies
- 0 kudos
- 0 kudos
The issue was most likely to use a CPU compute for the deployed model, switching to GPU (small) solved the issue.
- 0 kudos
- 8228 Views
- 8 replies
- 0 kudos
Feature Store Model Serving endpoint
Hi,I am trying to deploy my model which was logged by featureStoreEngineering client as a serving endpoint in Databricks. But I am facing following error: The Databricks Lookup client from databricks-feature-lookup and Databricks Feature Store clie...
- 8228 Views
- 8 replies
- 0 kudos
- 0 kudos
Hi @damselfly20 unfortunately I can't help much with that as I've never worked with RAGs. Are you sure it's the same error though? @NaeemS's and my errors seems to be Java related and yours MLflow related.
- 0 kudos
- 3014 Views
- 1 replies
- 1 kudos
Resolved! Vectorsearch ConnectionResetError Max retries exceeded
Hi,we are serving a unity catalog langchain model with databricks model serving. When I run the predict() function on the model in a notebook, I get the expected output. But when I query the served model, errors occur in the service logs:Error messag...
- 3014 Views
- 1 replies
- 1 kudos
- 1 kudos
downgrading langchain-community to version 0.2.4 solved my problem.
- 1 kudos
- 9193 Views
- 4 replies
- 3 kudos
Passing parameters in Databricks workflows
Hi Databricks, we have created several Databricks workflows and the `json-definition.json` for the same is stored inside version control i.e. GitHub. There are several parameters which are referred from params.json inside this job definition but the ...
- 9193 Views
- 4 replies
- 3 kudos
- 3 kudos
Have you considered using Databricks Asset Bundles? Very easy to parameterize!
- 3 kudos
- 6135 Views
- 4 replies
- 1 kudos
Resolved! Model flavour using feature store model training log_model()
Hi I'm have succesfully registered my model using the feature engineering client with the following codes:with mlflow.start_run(): # Calculate the ratio of negative class samples to positive class samples ratio = (len(y_train) - y_train.sum()...
- 6135 Views
- 4 replies
- 1 kudos
- 1 kudos
Thanks for your reply @robbe - yes I have created a custom pyfunc model which I can now use fe.score_batch() to return probabilities. Here is the code:# Calculate the ratio of negative class samples to positive class samples ratio = (len(y_train) - y...
- 1 kudos
- 6320 Views
- 2 replies
- 0 kudos
Can't load model from UC due to DBFS issue
I want to load a model I have registered in Unity Catalog using a Shared cluster, but it seems to be trying to use dbfs under the hood and it gives me an error.I am using DBR 13.3 LTS and mlflow-skinny[databricks]==2.14.3My code import mlflow mlflow...
- 6320 Views
- 2 replies
- 0 kudos
- 0 kudos
Have you tried to tell MLFlow to look for models in UC? mlflow.set_registry_uri("databricks-uc") Edit: never mind I see you have already. It shouldn't do/search for anything on DBFS anymore when setting this option so it is a bit strange. Shared clus...
- 0 kudos
- 1235 Views
- 0 replies
- 0 kudos
Creating an Input Schema for Multiple DataFrames in MLflow
Hi everyone,I am working with MLflow version 2.5.0 and need to create an input_schema for my model. My data schema is divided into several DataFrames, for example:{"dataframe_split": { "columns": ["ClientGuid", "Instance", "TypeScore", ...], ...
- 1235 Views
- 0 replies
- 0 kudos
- 6267 Views
- 4 replies
- 1 kudos
cluster sharing between different notebooks
I have two structured streaming notebooks running continuously for anomaly detection. Both notebooks import the same python module to mount the Azure blob storage, but each has its own container. Each notebook runs well when it has its own cluster. ...
- 6267 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @johnp , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback ...
- 1 kudos
- 4466 Views
- 3 replies
- 1 kudos
Attribute based access control in Unity catalog
Can I start using Attribute based access control ? Is it available now?
- 4466 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi, I want to use Attributed-Based Access Control, but I cannot find the option to create rules in my catalog. Is it already available in public preview?
- 1 kudos
- 27954 Views
- 4 replies
- 0 kudos
Resolved! databricks-cli
Hello! I am trying to use the databricks asset bundles through the webui on a databricks compute cluster. However to use this I need the databricks-cli library. I tried to install it on a cluster like described in the documentation using the curl com...
- 27954 Views
- 4 replies
- 0 kudos
- 0 kudos
Thank you for your help! I read over the part of the runtime of your cluster which has to be 15.0 or more in the documentation you linked. I checked and my compute was still on a LTS 14.3 runtime version, which was the cause.
- 0 kudos
