- 5336 Views
- 0 replies
- 0 kudos
computer vision
how does data bricks handle. computer vision related use cases? (eg defects detection for a manufacturing industry) is there a reference architecture
- 5336 Views
- 0 replies
- 0 kudos
Forum Posts
- 1829 Views
- 4 replies
- 1 kudos
Data AI Summit 2024
My first Data + AI summit and it's been a great experience
- 1829 Views
- 4 replies
- 1 kudos
Latest Reply
- 3245 Views
- 4 replies
- 0 kudos
Sharing Output between different tasks for MLOps pipeline as a Databricks Jobs
Hello EveryoneWe are trying to create an ML pipeline on Databricks using the famous Databricks workflows. Currently our pipeline includes having 3 major components: Data Ingestion, Model Training and Model Testing. My question is whether it is possib...
- 3245 Views
- 4 replies
- 0 kudos
Latest Reply
- 1776 Views
- 0 replies
- 0 kudos
Register Model mounted in S3
Hello!I'm having an issue registering a model saved in a mounted S3 bucket using mlflow.Let me give a little bit more context:1. First I mounted my S3 with all the corresponding IAM permissions:s3_bucket_name = f"s3a://{s3_bucket}"dbutils.fs.mount(so...
- 1776 Views
- 0 replies
- 0 kudos
- 4098 Views
- 3 replies
- 0 kudos
Resolved! Llm
Are LLMs really ready for production deployment?
- 4098 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
You should be careful while putting them to production without guardrails, perhaps using Mosaic AI gateway announced today that would aggregate these functionalities, it should be something to start. These are not the only things you should worry abo...
- 0 kudos
- 5772 Views
- 4 replies
- 4 kudos
Generate and export dbt documentation from the Workflow dbt task to S3
I'm testing the Databricks Jobs feature with a dbt task and wanted to know if you had any advice for me for managing dbt documentation.I can use "dbt run" commands to run my models then "dbt docs generate" to generate the documentation. But is it pos...
- 5772 Views
- 4 replies
- 4 kudos
Latest Reply
- 4 kudos
How can I access these target files from the task itself ? I am trying to use dbt's state modifiers for detecting models that changed and only running models when the source freshness changed. Is there an easy way to store and use these state files i...
- 4 kudos
- 1744 Views
- 1 replies
- 1 kudos
ML Model
What's the best option to store your trained ML models
- 1744 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
Depending on how many you have, different solutions may be appropriate - and conveniently, you can use MLflow as a front end for most of these if you're working in Python. If you're working on personal projects, a local MLflow instance might be the r...
- 1 kudos
- 1835 Views
- 2 replies
- 0 kudos
Can view Model Registry using a Service Principal, but cannot load the model for inference.
I have a Service Principal (for M2M auth) with read access to a Databricks Model Registry. I can successfully search the registry (via the `WorkspaceClient`) and find the model that I want to load using (Python) APIs, but I cannot load the model for ...
- 1835 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @JC3, Thank you for posting your question in the Databricks community. Is it possible to share with us the minimum reproducible code?
- 0 kudos
- 1707 Views
- 2 replies
- 0 kudos
Download model artifact with HTTP
Hi, I want to pass a link for Kserve to download a model registered in Mlflow, which uses an HTTP request method to do that (it can be downloaded directly from GitHub or HuggingFace). Will setting up an artifact store in S3 or other public storage se...
- 1707 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @leolmz, You can refer to the doc for downloading the model artifacts
- 0 kudos
- 2689 Views
- 1 replies
- 0 kudos
EasyOcr Endpoint not accepting inputs
Hi all! I am trying to create an endpoint for Easy OCR. I was able to create the experiment using a wrapper class with the code below: # import libraries import mlflow import mlflow.pyfunc import cloudpickle import cv2 import re import easyocr impo...
- 2689 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @John22, Thank you for posting your question on the Databricks community. First, are you able to infer the output within the notebook itself? Which cloud are you on AWS or Azure?
- 0 kudos
- 1064 Views
- 0 replies
- 0 kudos
AutoML models not completing
Hello, Whilst using a cluster set-up running 14.3 LTS ML, 2-10 workers, worker and driver type of r5d.xlarge I am having issues creating a regression model on 700k rows and 80 factors (no high cardinality in any factor shown).The first phase of the e...
- 1064 Views
- 0 replies
- 0 kudos
- 2857 Views
- 1 replies
- 0 kudos
DBRX - Serving endpoint failed - update timed out.
Hi,https://notebooks.databricks.com/demos/llm-rag-chatbot/index.htmlFollowing this tutorial I'm trying to serve an endpoint with DBRX model connected to my data in Vector Db.Without any problem I can log my model in Databricks with MLFlow and call th...
- 2857 Views
- 1 replies
- 0 kudos
- 5899 Views
- 1 replies
- 0 kudos
mlops-stacks workflow testing vs staging
I'm a newbie to MLOps and abit confused about the use and the implementation of staging and testing environment in the mlops-stack template. as far as I understand the staging environment is where we run the integration test. But in the ci-cd pipelin...
- 5899 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@MohsenJOfficial Site wrote:I'm a newbie to MLOps and abit confused about the use and the implementation of staging and testing environment in the mlops-stack template. as far as I understand the staging environment is where we run the integration te...
- 0 kudos
- 10208 Views
- 5 replies
- 5 kudos
DLT with unity catalog and ML
We are currently using DLT with unity catalog. DLT tables are created as materialized views in a schema inside a catalog. When we try to access these materialized view using a ML runtime (ex. 13.0 ML) cluster, it says, that we must use Single User se...
- 10208 Views
- 5 replies
- 5 kudos
Latest Reply
- 5 kudos
No updates as far as I am aware.You could make the workflow copying the data smart though and try to only do incremental updates, seems like a lot of effort though.
- 5 kudos
- 1452 Views
- 0 replies
- 0 kudos
Conditional email notification based on if statement - dbutils
Is it possible to trigger an email notification based on a conditional statement in Python without exiting the process?Specifically, I have a robustness check in my prediction pipeline that performs simple imputation when encountering missing data. T...
- 1452 Views
- 0 replies
- 0 kudos
