- 41446 Views
- 23 replies
- 51 kudos
Databricks Announces the Industry’s First Generative AI Engineer Learning Pathway and Certification
Today, we are announcing the industry's first Generative AI Engineer learning pathway and certification to help ensure that data and AI practitioners have the resources to be successful with generative AI. At Databricks, we recognize that generative ...
- 41446 Views
- 23 replies
- 51 kudos
- 1013 Views
- 1 replies
- 0 kudos
- 1013 Views
- 1 replies
- 0 kudos
- 0 kudos
For every endpoint there will be an inference table that can be configured to store the logs . https://docs.databricks.com/aws/en/ai-gateway/inference-tables#enable-and-disable-inference-tablesYou can then query and analyze this table - https://docs....
- 0 kudos
- 1901 Views
- 2 replies
- 0 kudos
Resolved! Agents button is missing
Dear Community,Recently I've followed the AI Agents on Databricks training and wanted to dive into AI world using my company Databricks account.After logging in to the workspace I was quite surprised to see that the Agents button in AI/ML menu is mis...
- 1901 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @kuznetsovvs , I checked this internally and on Azure, the Mosaic AI Agent Framework and Model Serving are supported in West Europe, the Agent Bricks UI itself is limited to specific regions (primarily in US). That's why the “Agents” button doesn’...
- 0 kudos
- 2476 Views
- 3 replies
- 1 kudos
Resolved! multi agent Genie AI app in serving endpoint fails with PERMISSION ERROR on Databricks tabls.
I have deployed simple multi agent genie app on serving endpoint and it always fails PERMISSION error on tables accessed by Genie agent, however the same app works perfectly when tested from notebook. not sure what am i missing here. I checked all th...
- 2476 Views
- 3 replies
- 1 kudos
- 1 kudos
hi @davidmorton thanks for responding, I am only using databricks_token for my authentication when tested via notebook. Can you help what changes needs to be done as per below notebook to enable access to tables when model deployed in serving endpoin...
- 1 kudos
- 1293 Views
- 1 replies
- 2 kudos
Resolved! How to create an information extraction agent (Trial Version)
I haven't been using Databricks for very long and I'm testing how things work. I tried to create a pre-built agent to extract information from one of my tables, but I get an error when I click create: "Permission denied: provisioned throughput is not...
- 1293 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Artur1 If you are using Agent Bricks to create the agent, this feature is not supported in the Databricks free edition.Please refer here Databricks Free Edition limitations
- 2 kudos
- 4695 Views
- 7 replies
- 1 kudos
Azure Content Understanding Equivalent
Hi all,I am exploring Databricks services or components that could be considered equivalent to Azure Document Intelligence and Azure Content Understanding.Our customer works with dozens of Excel and PDF files. These files follow multiple template typ...
- 4695 Views
- 7 replies
- 1 kudos
- 1 kudos
I would work, but you will need to specify and manage the ranges or number of header rows manually. You could potentially read the whole sheet in and then write some code that identifies the range that is of interest and cleans it in parsing. My reco...
- 1 kudos
- 3101 Views
- 1 replies
- 1 kudos
Databricks Vector Search Algorithm
I’m working on using Databricks vector search combined with full-text search in my application. I want to filter queries by the id field in my vector search index. I noticed that there is a limit of 1,024 IDs per query when using filters.If I need to...
- 3101 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @deane , I worked on a similar issue months earlier and the limit of 1024 is not configurable (no explicit way to increase the filter count), so my suggestion back then was to perform a two step process like filter for half of the filters once and...
- 1 kudos
- 3749 Views
- 1 replies
- 1 kudos
Resolved! Local LLM's available in Databricks for email classification
Hello everyone,I am currently working on an email classification model in Azure Databricks. Since I work for an international company, the emails contain PII data. Because of this, I need to be very careful about compliance and data privacy, especial...
- 3749 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, It is absolutely acceptable. Here are some details that you may want to consider. I'd also think about GPU availability in your cloud and region and whether there is GPU available for you to deploy these models to. You should be able to easily te...
- 1 kudos
- 3633 Views
- 4 replies
- 3 kudos
Resolved! Accessing Knowledge Base from Databricks One
Is it possible to use Knowledge Assistant from Databricks one ?
- 3633 Views
- 4 replies
- 3 kudos
- 3 kudos
@piotrsofts , if you are happy please accept as a solution so others can be confident in the approach. Cheers, Louis.
- 3 kudos
- 1244 Views
- 2 replies
- 1 kudos
Resolved! Not able to add scorer to multi agent supervisor
Hello,When I try to add scorers to Multi agent endpoint based on the last 10 traces that I have logged and visible in the experiments tab, i get this error.Also, are there any demos which i can refer regarding the tabs within the evaluation bar expla...
- 1244 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @shivamrai162 , Did you add the last 10 traces to the evaluation dataset? You can follow the steps here to make sure you added the traces to the evaluation dataset. To answer your second question, here is a good article that covers the concepts an...
- 1 kudos
- 2300 Views
- 1 replies
- 1 kudos
Resolved! Claude Code-Execution Call Format
Is it possible to use the code execution tool with instances of Claude hosted through Databricks? If I try to format the payload like in Anthropic's documentation, I get an error that the function isn't defined properly: tools=[{ "type": ...
- 2300 Views
- 1 replies
- 1 kudos
- 1 kudos
Greeting @jmartin1 , The Anthropic “code execution” tool isn’t supported through Databricks’ Foundation Model APIs. Databricks exposes an OpenAI-compatible tools interface, and the only supported tool type today is a function defined with a JSON sch...
- 1 kudos
- 11390 Views
- 3 replies
- 1 kudos
Behavior of Vector Index Sync with Delta Tables When Using OVERWRITE vs MERGE in Databricks
I'm working with vector search in Databricks using vector index sync with Delta tables, and I'm a bit unclear on how updates to the source table affect the vector index, specifically when using different write operations.If I overwrite the source Del...
- 11390 Views
- 3 replies
- 1 kudos
- 1 kudos
From community experience, vector index sync behavior depends heavily on how the Delta table is updated. With OVERWRITE, the table is effectively replaced, so the vector index typically treats this as a full refresh. Existing embeddings are dropped a...
- 1 kudos
- 1516 Views
- 3 replies
- 2 kudos
Resolved! How do I integrate third-party ML/AI libraries with Databricks GenAI workflows?
How can I use external AI libraries inside my Databricks GenAI projects?
- 1516 Views
- 3 replies
- 2 kudos
- 2 kudos
You can use third-party ML/AI libraries in Databricks GenAI by installing them on your cluster or notebook (%pip install library-name), importing them in your code, and then integrating their outputs into your GenAI workflows. For large models, use a...
- 2 kudos
- 2691 Views
- 5 replies
- 2 kudos
ai_parse_document + Genie with ai_query
Hi EveryoneI have used ai_parse_document to process multiple PDFs and store the parsed data in a table (one PDF per row). Later, I ran ai_query in natural language, which correctly scans all rows and returns answers from each PDF.However, when I use ...
- 2691 Views
- 5 replies
- 2 kudos
- 2 kudos
I think Genie is not optimized for this use case. Please run some experiments - chunk pdf to use multi-agent supervisor from agent bricks to combine Genie with Knowledge Base (although I haven't yet it yet)
- 2 kudos
- 2191 Views
- 1 replies
- 0 kudos
Resolved! What’s the recommended architecture for integrating Databricks + MLflow + GenAI for model training
If I want to use Databricks, MLflow, and GenAI together, what is the best way to organize and connect them so that I can train AI models and then use them in real apps?
- 2191 Views
- 1 replies
- 0 kudos
- 0 kudos
Broad question. I will recommend to follow Databricks offical documents and ML Training at their customer portal.Yet, I will try to answer it as belowUse the Databricks Lakehouse as the unified platform, with Unity Catalog (UC) providing centralized ...
- 0 kudos
- 8366 Views
- 6 replies
- 3 kudos
Resolved! ai_parse_document struggling to detect pdf
Hi helpful experts I'm writing my first PySpark Notebook that makes use of the new `ai_parse_document` function. I am basically following the code example from here: https://learn.microsoft.com/en-gb/azure/databricks/sql/language-manual/functions/ai...
- 8366 Views
- 6 replies
- 3 kudos
- 3 kudos
Hello @JN_Bristol,I discovered that ai_parse_document only works when the input is parsed as real Python bytes.The binaryFile format in Spark returns the content as an internal binary type (like a memoryview), and ai_parse_document can’t process that...
- 3 kudos
