- 33509 Views
- 22 replies
- 46 kudos
Databricks Announces the Industry’s First Generative AI Engineer Learning Pathway and Certification
Today, we are announcing the industry's first Generative AI Engineer learning pathway and certification to help ensure that data and AI practitioners have the resources to be successful with generative AI. At Databricks, we recognize that generative ...
- 33509 Views
- 22 replies
- 46 kudos
- 46 kudos
Dear Certifications TeamI have completed full Generative AI Engineering Pathway, so I received module wise knowledge badge but I didn't received the overall certificate which mentioned in description which is Generative AI Engineer with one Star. Req...
- 46 kudos
- 4910 Views
- 1 replies
- 2 kudos
AI/BI Genie - Real world experience?
I have been tasked with creating a NLP interface for management to our internal datalake that has numerous joined tables around customer information. I have tried the hello-world llamaindex, langchain sql/pandas agents with limited success (too many ...
- 4910 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi there Sam,I work for Databricks. The pace of innovation with AI/BI is wild (fast). Databricks itself just migrated off a leading BI platform and now exclusively runs its business on AI/BI (Blog). We also have customer testimonials on our website, ...
- 2 kudos
- 4405 Views
- 5 replies
- 3 kudos
Using Genie in Multi-Agent Systems — SQL Query Missing from Output
Hi Community,I’ve implemented a multi-agent system using Genie in Databricks, following the guide on "Use Genie in multi-agent systems | Databricks Documentation". The system integrates well with vector indexes, and everything works fine during model...
- 4405 Views
- 5 replies
- 3 kudos
- 3 kudos
When using a custom Genie agent with the Databricks Genie REST API (GetMessage endpoint), extracting SQL queries and results works well as a standalone agent. However, integrating this custom agent into a multi-agent framework (especially when combin...
- 3 kudos
- 5460 Views
- 2 replies
- 0 kudos
DatabricksFunctionClient : PERMISSION_DENIED when creating the client
Hello, I'm currently working with Databricks features to create an AI agent system. In the documentation it's written that the library DatabricksFunctionClient is needed to register custom tools to UC. I followed the different steps from the explana...
- 5460 Views
- 2 replies
- 0 kudos
- 0 kudos
This error is most often due to missing permissions, workspace types, or cluster configuration settings necessary for AI agents and custom tools to access Unity Catalog (UC) functions using Spark Connect in Databricks. The main causes and solutions a...
- 0 kudos
- 6534 Views
- 3 replies
- 2 kudos
Langchain SQLDatabase not fetching column names from table in Unity Catalog
I am building a text-to-sql agent using Langchain API.I created a SQLDatabase using: db = SQLDatabase.from_databricks(catalog="`my-catalog-name`", schema="my_schema", host="...", api_token="...", warehouse_id="12345678")When I call db.get_table_info(...
- 6534 Views
- 3 replies
- 2 kudos
- 2 kudos
The issue happens because SQLDatabase.from_databricks doesn’t automatically fetch column metadata from Unity Catalog tables in some LangChain versions, leading to malformed SELECT statements. A workaround is to manually query the catalog like you did...
- 2 kudos
- 7696 Views
- 1 replies
- 1 kudos
Advanced RAG Retrieval (Reranking, Hierarchical, etc) in Databricks
Issue with current documentation:I wish to perform advanced RAG using langchain, in Databricks. In the documentation, they tell how to use the vector endpoint url, and index stored in catalogs. But I could not find any advanced RAG algos that are eas...
- 7696 Views
- 1 replies
- 1 kudos
- 1 kudos
Greetings @meetiasha , yes—there’s a gap between Databricks’ basic “vector endpoint + catalog index” examples and truly advanced RAG, so below is a step‑wise, LangChain‑first playbook you can run entirely on Databricks notebooks with local vector sto...
- 1 kudos
- 4917 Views
- 1 replies
- 1 kudos
GenAI Cookbook - how to add source documents to output and open pdf file on a page
Hello,I am implementing RAG solution as per Databricks cookbook. Review App is working, references are provided as text chunks.I need to build functionality to open pdf file on a specific page as a reference. Is there a way to change ReviewApp to ope...
- 4917 Views
- 1 replies
- 1 kudos
- 1 kudos
You want your RAG solution (based on Databricks Cookbook) to display PDF files at specific pages as references in your Review App, rather than plain text chunks. You also wish to retrieve source documents from your serving endpoint, but your current ...
- 1 kudos
- 5348 Views
- 1 replies
- 0 kudos
Invoke Azure AI Search Endpoint through Databricks environment
Hi Team,Is there a possibility to invoke Azure AI Search Vector DB endpoint(external) in Databricks environment based on input data in Databricks table.Scenario: Client-Specific documents are already embedded in Azure AI Search Vector DB. Is there an...
- 5348 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, it is possible to invoke an Azure AI Search Vector DB endpoint from within a Databricks environment—allowing you to leverage your existing Azure resource for client-specific document retrieval, without needing to create a new vector database in ...
- 0 kudos
- 7183 Views
- 6 replies
- 3 kudos
Resolved! How to integrate genie in with databricks apps
How to integrate genie in with databricks - data apps i want a chatbot that behaves like genie
- 7183 Views
- 6 replies
- 3 kudos
- 3 kudos
Thank you for the code. However, when user asks a follow up question, app responds with irrelevant data. The context or history is not preserved. How to handle this?
- 3 kudos
- 1975 Views
- 1 replies
- 0 kudos
Deploying HuggingFace LLM model with MLflow task llm/v1/chat into Databricks
Hello,I am currently trying to deploy a HuggingFace LLM model to Databricks with the MLflow task llm/v1/chat in order to use it as a chat.I have tried several models like:TinyLlama/TinyLlama_v1.1 · Hugging FaceBSC-LT/salamandra-7b-instruct · Hugging ...
- 1975 Views
- 1 replies
- 0 kudos
- 0 kudos
Deploying HuggingFace LLM models to Databricks using MLflow’s llm/v1/chat task sometimes results in unexpected chat behaviors, usually due to prompt/template mismatches, model configuration issues, or pipeline setup requirements. Here’s a direct answ...
- 0 kudos
- 6180 Views
- 1 replies
- 0 kudos
Issue in creating endpoint for quantized gguf model with llama-cpp-python
Hello, Databricks Community,I am experiencing an issue while trying to serve a quantized model in gguf format using Databricks serving with the llama-cpp-python library.The model is registered using MLflow and pyfunc on Unity. The model loads without...
- 6180 Views
- 1 replies
- 0 kudos
- 0 kudos
The error code 132 typically means an illegal instruction was encountered, often caused by a CPU incompatibility with the code being executed—especially with libraries that use SIMD or hardware acceleration (e.g., llama-cpp-python, which is often com...
- 0 kudos
- 4855 Views
- 1 replies
- 0 kudos
Not able to invoke model external model
I have followed below steps1) Created serving end point by for external model gpt-4-turbo and providing azure AI endpoint and key2) Now using langchain, i am trying to connect and invoke message from model in notebook model = ChatDatabricks(target_u...
- 4855 Views
- 1 replies
- 0 kudos
- 0 kudos
Based on your description, you are encountering a 500 Server Error when trying to use the Langchain ChatDatabricks integration with a Databricks Serving Endpoint connected to an external OpenAI GPT-4 Turbo model on Azure. This error usually indicates...
- 0 kudos
- 5404 Views
- 1 replies
- 0 kudos
Mlflow.evaluation fails to generate score
The execution of code stucks when evaluation of data start. eval_df = pd.DataFrame( { "inputs": [ "What is MLflow?", "What is Spark?", ], "ground_truth": [ "MLflow is an open-source platform f...
- 5404 Views
- 1 replies
- 0 kudos
- 0 kudos
The issue described—a Databricks notebook getting "stuck" during the evaluation phase using mlflow.evaluate—is most likely related to environment setup, model compatibility, or limitations with the mlflow.pyfunc.log_model and the evaluation utilities...
- 0 kudos
- 5599 Views
- 1 replies
- 0 kudos
I need a sample code or process which will help us to dynamically select the prompt template
We need a sample code or process which will help us to dynamically select the prompt template based on the prompt given as an input through the model legacy serving endpoint
- 5599 Views
- 1 replies
- 0 kudos
- 0 kudos
To dynamically select a prompt template in Databricks based on the input prompt received through a legacy model serving endpoint, you can implement a Python function that maps incoming prompts to specific templates. This often involves using conditio...
- 0 kudos
- 7016 Views
- 1 replies
- 0 kudos
How to perform combined search on structured and unstructured data in databrick using RAG or other
I created a RAG application in databricks which performs the following steps:1. Extract text from PDF files2. Prepare embeddings on extracted text and create vector search index3. Create a LLM model and served the model which can answer question base...
- 7016 Views
- 1 replies
- 0 kudos
- 0 kudos
You can achieve combined retrieval across both PDF-extracted unstructured data and multiple columns from structured Delta tables in Databricks, but there are important considerations and available patterns to optimize this workflow for your RAG appli...
- 0 kudos
- 4635 Views
- 1 replies
- 0 kudos
Issue with Multi agent supervisor based agentic framework
I have three agents in my multi agent framework (code attached) Supervisor : This is the main controller "Genie": #this is a genai agent "Coder": #this is a re-act agent created on the fly "Weather": #this is an exis...
- 4635 Views
- 1 replies
- 0 kudos
- 0 kudos
Your multi-agent framework has several issues affecting agent iteration control and error handling. Here’s a lined explanation and practical suggestions for each problem: Agents not stopping at one iteration Agents consistently reaching the max iter...
- 0 kudos
