- 84 Views
- 1 replies
- 0 kudos
Interactive EDA task in a Job Workflow
I am trying to configure an interactive EDA task as part of a job workflow. I'd like to be able to trigger a workflow, perform some basic analysis then proceed to a subsequent task. I haven't had any success freezing execution. Also, the job workflow...
- 84 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @cmd0160, Freezing job execution to perform interactive tasks directly within a job workflow is not natively supported in Databricks. The job workflow UI and the notebook UI serve different purposes, and the interactive capabilities you find in...
- 0 kudos
- 404 Views
- 0 replies
- 0 kudos
Patient Risk Score based on health history: Unable to create data folder for artifacts in S3 bucket
Hi All,we're using the below git project to build PoC on the concept of "Patient-Level Risk Scoring Based on Condition History": https://github.com/databricks-industry-solutions/hls-patient-riskI was able to import the solution into Databricks and ru...
- 404 Views
- 0 replies
- 0 kudos
- 541 Views
- 0 replies
- 0 kudos
Learn Databricks AI medium article series for fellow learners.
When it comes to machine learning, the platform plays a pivotal role in successful implementation. Databricks offers a best-in-class machine learning platform with cutting-edge features such as MLflow, Model Registry, Feature Store, and MLOps, which ...
- 541 Views
- 0 replies
- 0 kudos
- 905 Views
- 3 replies
- 0 kudos
Consequences of Not Using write_table with Feature Engineering Client and INSERT OVERWRITE
Hello Databricks Community,I am currently using the Feature Engineering client and have a few questions about best practices for writing to Feature Store Tables.I would like to know more about not using the write_table method directly from the featur...
- 905 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @zed,How are you doing? As per my understanding, Consider using the write_table method from the Feature Engineering client to ensure that all Feature Store functionality is properly leveraged, such as cataloging, lineage tracking, and handling upd...
- 0 kudos
- 732 Views
- 0 replies
- 0 kudos
FeatureEngineeringClient and Unity Catalog
When testing this code ( fe.score_batch( df=dataset.drop("Target").limit(10), model_uri=f"models:/{model_name}/{mv.version}", ) .select("prediction") .limit(10) .display() ) I get the error: “MlflowException: The...
- 732 Views
- 0 replies
- 0 kudos
- 598 Views
- 1 replies
- 1 kudos
Feature Store - lookback_window does not work with primary keys of "date" type
I just discovered what I believe is a bug in Feature Store. The expected value (of the "value" column) is 'NULL' but the actual value is "a". If I instead change the format to timestamp of the "date" column (i.e. removes the .date() in the generation...
- 598 Views
- 1 replies
- 1 kudos
- 1 kudos
Thank you for answering. Yes, that is also what I figured out. In other words the lookback_window argument only works when using timestamp format for the primary key. I cannot see that this behavior is described in the documentation.
- 1 kudos
- 3728 Views
- 8 replies
- 0 kudos
Feature Store Model Serving endpoint
Hi,I am trying to deploy my model which was logged by featureStoreEngineering client as a serving endpoint in Databricks. But I am facing following error: The Databricks Lookup client from databricks-feature-lookup and Databricks Feature Store clie...
- 3728 Views
- 8 replies
- 0 kudos
- 0 kudos
Hi @damselfly20 unfortunately I can't help much with that as I've never worked with RAGs. Are you sure it's the same error though? @NaeemS's and my errors seems to be Java related and yours MLflow related.
- 0 kudos
- 2441 Views
- 2 replies
- 0 kudos
Model Serving Latency Chart
Hi, For the model serving latency graph what is p50 and p99? I only have one model i am serving on this endpoing so im surprised to see two models being tracked
- 2441 Views
- 2 replies
- 0 kudos
- 0 kudos
If im not mistaken this refers to 50% of responses and 99% responses and averages accordingly for the metrics? @s_park @Sujitha @Debayan
- 0 kudos
- 1951 Views
- 3 replies
- 0 kudos
Import mlflow Error
Hello, I am trying to replicate this motebook in my environment: mlflow-end-to-end-example - Databricks However, I am getting the following error when I run "import mlflow": "TypeError: bases must be types"How can I solve this issue? Thank you, Tanji...
- 1951 Views
- 3 replies
- 0 kudos
- 0 kudos
Hello @tanjil Thank you for contacting databricks community support. Could you check what version of protobuf you have? If you are using 10.4 ML cluster, the MLflow 1.x is not compatible with protobuf 4.x. The default version of protobuf in MLR 10...
- 0 kudos
- 2416 Views
- 0 replies
- 0 kudos
'error_code': 'INVALID_PARAMETER_VALUE', 'message': 'Too many sources. It cannot be more than 100'
I am getting the following error while saving a delta table in the feature storeWARNING databricks.feature_store._catalog_client_helper: Failed to record data sources in the catalog. Exception: {'error_code': 'INVALID_PARAMETER_VALUE', 'message': 'To...
- 2416 Views
- 0 replies
- 0 kudos
- 2303 Views
- 1 replies
- 0 kudos
Feature Store Log Model and Score Batch - env_manager
Hi Everyone. I have a couple of questions about the feature store log model and score batch. After you log a model with the feature store then use fs.score_batch is it possible to pass the env_manager to predict with the same env as training as descr...
- 2303 Views
- 1 replies
- 0 kudos
- 0 kudos
I also like to know if that works.
- 0 kudos
- 1746 Views
- 2 replies
- 1 kudos
Model Lineage with Feature Engineering is missing tables and notebooks
I am trying to track the lineage of model and tables using the FeatureEngineeringClient. The table lineage shows the relevant tables and notebooks but the model lineage shows only the model. No notebook and tables. here is my code fe = FeatureEngine...
- 1746 Views
- 2 replies
- 1 kudos
- 1 kudos
ok I realized something else. That although I used FeatureEngineeringCient, MLflow model artifact suggest I used FeatureStoreClient. Please see attachment.
- 1 kudos
- 2051 Views
- 0 replies
- 1 kudos
Handling Null Values in Feature Stores
Hi, I am using multiple feature stores in my workflow using feature lookups. In my logged pipeline, I have several stages, including Assembler, Standard Scaler, Indexer and then Model. However, I am facing an issue during inference using the `score b...
- 2051 Views
- 0 replies
- 1 kudos
- 4129 Views
- 5 replies
- 5 kudos
Does FeatureStoreClient().score_batch support multidimentional predictions?
I have a pyfunc model that I can use to get predictions. It takes time series data with context information at each date, and produces a string of predictions. For example:The data is set up like below (temp/pressure/output are different than my inpu...
- 4129 Views
- 5 replies
- 5 kudos
- 5 kudos
I have the same question. I've decided to look for alternative Feature Stores as this makes it very difficult to use for time series forecasting.
- 5 kudos
- 5318 Views
- 2 replies
- 2 kudos
Resolved! Differences between Feature Store and Unity Catalog
Our small team has just finished the data preparation phase of our project and started data analysis in Databricks. As we go deeper into this field, we're trying to understand the distinctions and appropriate uses for a Feature Store versus a Unity C...
- 5318 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Northp Good day!1.) A Feature Store is a centralized repository that enables data scientists to find and share features, ensuring that the same code used to compute the feature values is used for model training and inference. It is particularly...
- 2 kudos