- 179 Views
- 1 replies
- 0 kudos
Serving model with custom scoring script to a real-time endpoint
Hi, new to databricks here and wasn't able to find relevant info in the documentation.Is it not possible to serve a model with a custom scoring script to an online endpoint on databricks to customise inference ? the customisation is related to incomi...
- 179 Views
- 1 replies
- 0 kudos
- 0 kudos
If I'm understanding, all you really want to do is have a pre/post - process function running with your model, is that correct? If so, you can do this by using the MLflow pyfunc model. Something like they do here:https://docs.databricks.com/en/machi...
- 0 kudos
- 141 Views
- 0 replies
- 1 kudos
Table-Model Lineage for models without online Feature Lookups
Hi community,I am looking for the recommended way to achieve table-model lineage in Unity Catalog for models that don't use Feature Lookups but only offline features. When I use FeatureEngineeringClient.create_training_set with feature_lookups + mlfl...
- 141 Views
- 0 replies
- 1 kudos
- 2255 Views
- 3 replies
- 1 kudos
Resolved! Model Serving Endpoint Creation through API
Hello,I am trying to create a model serving endpoint via the API as explained here: https://docs.databricks.com/api/workspace/servingendpoints/createI created a trusted IAM role with access to DynamoDB for the feature store. I try to use this field,"...
- 2255 Views
- 3 replies
- 1 kudos
- 1 kudos
I have a similar issue with a model serving endpoint I created for serving external models (Amazon Bedrock) with Mosaic AI model serving (https://docs.databricks.com/en/generative-ai/external-models/index.html). For this also, I can use an instance p...
- 1 kudos
- 560 Views
- 3 replies
- 0 kudos
Consequences of Not Using write_table with Feature Engineering Client and INSERT OVERWRITE
Hello Databricks Community,I am currently using the Feature Engineering client and have a few questions about best practices for writing to Feature Store Tables.I would like to know more about not using the write_table method directly from the featur...
- 560 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @zed,How are you doing? As per my understanding, Consider using the write_table method from the Feature Engineering client to ensure that all Feature Store functionality is properly leveraged, such as cataloging, lineage tracking, and handling upd...
- 0 kudos
- 127 Views
- 0 replies
- 0 kudos
Hyperopt (15.4 LTS ML) ignores autologger settings
I use ML Flow Experiment to store models once they leave very early tests and development. I switched lately to 15.4 LTS ML and was hit by unhinged Hyperopt behavior:it was creating Experiment logs ignoring i) autologger is off on the workspace level...
- 127 Views
- 0 replies
- 0 kudos
- 136 Views
- 0 replies
- 2 kudos
Bug: MLflow recipe
I'm not sure whether this is the right place, but we've encountered a bug in the datasets.py(https://github.com/mlflow/mlflow/blob/master/mlflow/recipes/steps/ingest/datasets.py.). Anyone using recipes beware of forementioned.def _convert_spark_df_to...
- 136 Views
- 0 replies
- 2 kudos
- 283 Views
- 2 replies
- 0 kudos
AutoML forecast only supports integers as predicate target ?
Hi Community,I've playing around with AutoML and started with a simple forecast for Databricks samples.I used a copy of table samples.tpch.orders.To my supprise only integer types were available as Predicat Target. The field I was interested in forec...
- 283 Views
- 2 replies
- 0 kudos
- 0 kudos
@jkibiki wrote:Hi Community,I've playing around with AutoML and started with a simple forecast for Databricks samples.I used a copy of table samples.tpch.orders.To my supprise only integer types were available as Predicat Target. The field I was int...
- 0 kudos
- 1003 Views
- 3 replies
- 1 kudos
Not able to edit_mode UI_LOCKED to EDITABLE in bundle deployment for development mode
The edit_mode for Databricks jobs cannot be overridden using the bundle. Based on the jobs REST API docs, there is a functionality to set this parameter but in the bundle docs, it's not available. How can I use this in the bundle to override the para...
- 1003 Views
- 3 replies
- 1 kudos
- 1 kudos
The `edit_mode` property cannot be set by design. It is set to `UI_LOCKED` on bundle deployment to let viewers of the job in the UI know that any changes they make to the job instance are going to be clobbered the next time someone runs a bundle depl...
- 1 kudos
- 3189 Views
- 3 replies
- 2 kudos
How to proper use Databricks MLFlow Managed tracker/register with Databricks Workflow
Hey.I'm building a DevOps/MLOps pipeline to train/register simple scikit learn model.I created a simple Databricks Workflow to execute training and register task on specific .git branch. (Workflow is setup with Databricks Repo on specifc branch, with...
- 3189 Views
- 3 replies
- 2 kudos
- 2 kudos
I had same issue while trying to call notebook from workflow. I was able to do what you did. But it needs new experiment name for each run, so I had to do this:# Set the experimentexperiment_name = f"/Workspace/MLOps/{env}/experiment/{experiment}_{ti...
- 2 kudos
- 2064 Views
- 1 replies
- 2 kudos
Run mlflow project from a Job.
Hey Guys, I'm trying to make automated process to run ML training sessions using mlflow and databricks jobs.While developing the model on my local machine using IDE, When finished I have a template notebook that get as parameters the mlflow project p...
- 2064 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi,Were you able to figure out this one? I have same issue trying to call training notebook from workflow. Each run needs a new experiment name which I can do but then it creates a new experiment ID/name for each workflow run. Where as when you run f...
- 2 kudos
- 314 Views
- 1 replies
- 0 kudos
Serving Endpoint Deployment
Hello Community,I am seeking assistance with an issue related to serving a custom Hugging Face model (M2M100). I successfully registered the model in Unity Catalog using the MLflow Python libraries without any problems. However, when attempting to se...
- 314 Views
- 1 replies
- 0 kudos
- 0 kudos
see https://community.databricks.com/t5/machine-learning/serving-endpoint-container-image-creation-fails/td-p/83458
- 0 kudos
- 1052 Views
- 2 replies
- 0 kudos
Azure databricks API and DLT databricks
how can i pass parameter from Azure data factory rest web API to delta live Databricks?I get this error: "Py4JJavaError: An error occurred while calling o382.getArgument.: com.databricks.dbutils_v1.InputWidgetNotDefined: No input widget named *** def...
- 1052 Views
- 2 replies
- 0 kudos
- 0 kudos
@girl_chan The error you are encountering usually occurs when a widget referenced in a Databricks notebook is not defined. In your case, the issue is likely related to how you pass parameters from Azure Data Factory (ADF) to Delta Live Tables (DLT) i...
- 0 kudos
- 1549 Views
- 2 replies
- 0 kudos
Rollback cluster changes
Is it possible to rollback changes made to a cluster? The problem I'm trying to solve is to recover from an accidental change made by a user on a cluster that affects interactive and job runs. Cluster policies help, but the policy still provides the ...
- 1549 Views
- 2 replies
- 0 kudos
- 0 kudos
@User16826990884 Along with what @sajith_appukutt mentioned, we can achive this viaVersion Control for Cluster Configurations: Store cluster configurations in JSON files in GitHub or another version control system.In case of accidental changes, you c...
- 0 kudos
- 230 Views
- 0 replies
- 0 kudos
Custom AutoML pipeline: Beyond StandardScaler().
The automated notebook pipeline in an AutoML experiment applies StandardScaler to all numerical features in the training dataset as part of the PreProcessor. See below.But I want a more nuanced and varied treatment of my numeric values (e.g. I have l...
- 230 Views
- 0 replies
- 0 kudos
- 250 Views
- 0 replies
- 0 kudos
Surprisingly sparse_logs and tensorboard logfiles in Databricks-Workspace
Hi, surprisingly we have found 2 new folders with some short logfiles in our Databricks workspace:ls -lFr sparse_logs/ tensorboard/tensorboard/:-rwxrwxrwx 1 root root 88 Sep 2 11:26 events.out.tfevents.1725275744.0830-063833-n68nsxoq-10-139-64-10.20...
- 250 Views
- 0 replies
- 0 kudos
