- 35376 Views
- 22 replies
- 47 kudos
Databricks Announces the Industry’s First Generative AI Engineer Learning Pathway and Certification
Today, we are announcing the industry's first Generative AI Engineer learning pathway and certification to help ensure that data and AI practitioners have the resources to be successful with generative AI. At Databricks, we recognize that generative ...
- 35376 Views
- 22 replies
- 47 kudos
- 47 kudos
Dear Certifications TeamI have completed full Generative AI Engineering Pathway, so I received module wise knowledge badge but I didn't received the overall certificate which mentioned in description which is Generative AI Engineer with one Star. Req...
- 47 kudos
- 4952 Views
- 1 replies
- 0 kudos
Unable to add dependencies to mlflow.langchain.log_model
Hello,I'm doing this-with mlflow.start_run(run_name="run1"): logged_chain_info = mlflow.langchain.log_model( # lc_model=os.path.join(os.getcwd(), 'full_chain'), this doesnt work too lc_model='/Workspace/Users/{user_name}/exp/d...
- 4952 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @yashshingvi , Thanks for the details—this is a common gotcha with MLflow “models from code.” Why your imports fail code_paths are only added to sys.path when the model is loaded (for inference/serving), not while the driver is executing mlflow.la...
- 0 kudos
- 5144 Views
- 1 replies
- 2 kudos
AI/BI Genie - Real world experience?
I have been tasked with creating a NLP interface for management to our internal datalake that has numerous joined tables around customer information. I have tried the hello-world llamaindex, langchain sql/pandas agents with limited success (too many ...
- 5144 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi there Sam,I work for Databricks. The pace of innovation with AI/BI is wild (fast). Databricks itself just migrated off a leading BI platform and now exclusively runs its business on AI/BI (Blog). We also have customer testimonials on our website, ...
- 2 kudos
- 5299 Views
- 5 replies
- 3 kudos
Using Genie in Multi-Agent Systems — SQL Query Missing from Output
Hi Community,I’ve implemented a multi-agent system using Genie in Databricks, following the guide on "Use Genie in multi-agent systems | Databricks Documentation". The system integrates well with vector indexes, and everything works fine during model...
- 5299 Views
- 5 replies
- 3 kudos
- 3 kudos
When using a custom Genie agent with the Databricks Genie REST API (GetMessage endpoint), extracting SQL queries and results works well as a standalone agent. However, integrating this custom agent into a multi-agent framework (especially when combin...
- 3 kudos
- 5759 Views
- 2 replies
- 0 kudos
DatabricksFunctionClient : PERMISSION_DENIED when creating the client
Hello, I'm currently working with Databricks features to create an AI agent system. In the documentation it's written that the library DatabricksFunctionClient is needed to register custom tools to UC. I followed the different steps from the explana...
- 5759 Views
- 2 replies
- 0 kudos
- 0 kudos
This error is most often due to missing permissions, workspace types, or cluster configuration settings necessary for AI agents and custom tools to access Unity Catalog (UC) functions using Spark Connect in Databricks. The main causes and solutions a...
- 0 kudos
- 7309 Views
- 3 replies
- 2 kudos
Langchain SQLDatabase not fetching column names from table in Unity Catalog
I am building a text-to-sql agent using Langchain API.I created a SQLDatabase using: db = SQLDatabase.from_databricks(catalog="`my-catalog-name`", schema="my_schema", host="...", api_token="...", warehouse_id="12345678")When I call db.get_table_info(...
- 7309 Views
- 3 replies
- 2 kudos
- 2 kudos
The issue happens because SQLDatabase.from_databricks doesn’t automatically fetch column metadata from Unity Catalog tables in some LangChain versions, leading to malformed SELECT statements. A workaround is to manually query the catalog like you did...
- 2 kudos
- 9112 Views
- 1 replies
- 1 kudos
Advanced RAG Retrieval (Reranking, Hierarchical, etc) in Databricks
Issue with current documentation:I wish to perform advanced RAG using langchain, in Databricks. In the documentation, they tell how to use the vector endpoint url, and index stored in catalogs. But I could not find any advanced RAG algos that are eas...
- 9112 Views
- 1 replies
- 1 kudos
- 1 kudos
Greetings @meetiasha , yes—there’s a gap between Databricks’ basic “vector endpoint + catalog index” examples and truly advanced RAG, so below is a step‑wise, LangChain‑first playbook you can run entirely on Databricks notebooks with local vector sto...
- 1 kudos
- 5166 Views
- 1 replies
- 1 kudos
GenAI Cookbook - how to add source documents to output and open pdf file on a page
Hello,I am implementing RAG solution as per Databricks cookbook. Review App is working, references are provided as text chunks.I need to build functionality to open pdf file on a specific page as a reference. Is there a way to change ReviewApp to ope...
- 5166 Views
- 1 replies
- 1 kudos
- 1 kudos
You want your RAG solution (based on Databricks Cookbook) to display PDF files at specific pages as references in your Review App, rather than plain text chunks. You also wish to retrieve source documents from your serving endpoint, but your current ...
- 1 kudos
- 5761 Views
- 1 replies
- 0 kudos
Invoke Azure AI Search Endpoint through Databricks environment
Hi Team,Is there a possibility to invoke Azure AI Search Vector DB endpoint(external) in Databricks environment based on input data in Databricks table.Scenario: Client-Specific documents are already embedded in Azure AI Search Vector DB. Is there an...
- 5761 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, it is possible to invoke an Azure AI Search Vector DB endpoint from within a Databricks environment—allowing you to leverage your existing Azure resource for client-specific document retrieval, without needing to create a new vector database in ...
- 0 kudos
- 7889 Views
- 6 replies
- 3 kudos
Resolved! How to integrate genie in with databricks apps
How to integrate genie in with databricks - data apps i want a chatbot that behaves like genie
- 7889 Views
- 6 replies
- 3 kudos
- 3 kudos
Thank you for the code. However, when user asks a follow up question, app responds with irrelevant data. The context or history is not preserved. How to handle this?
- 3 kudos
- 2524 Views
- 1 replies
- 0 kudos
Deploying HuggingFace LLM model with MLflow task llm/v1/chat into Databricks
Hello,I am currently trying to deploy a HuggingFace LLM model to Databricks with the MLflow task llm/v1/chat in order to use it as a chat.I have tried several models like:TinyLlama/TinyLlama_v1.1 · Hugging FaceBSC-LT/salamandra-7b-instruct · Hugging ...
- 2524 Views
- 1 replies
- 0 kudos
- 0 kudos
Deploying HuggingFace LLM models to Databricks using MLflow’s llm/v1/chat task sometimes results in unexpected chat behaviors, usually due to prompt/template mismatches, model configuration issues, or pipeline setup requirements. Here’s a direct answ...
- 0 kudos
- 7166 Views
- 1 replies
- 0 kudos
Issue in creating endpoint for quantized gguf model with llama-cpp-python
Hello, Databricks Community,I am experiencing an issue while trying to serve a quantized model in gguf format using Databricks serving with the llama-cpp-python library.The model is registered using MLflow and pyfunc on Unity. The model loads without...
- 7166 Views
- 1 replies
- 0 kudos
- 0 kudos
The error code 132 typically means an illegal instruction was encountered, often caused by a CPU incompatibility with the code being executed—especially with libraries that use SIMD or hardware acceleration (e.g., llama-cpp-python, which is often com...
- 0 kudos
- 5144 Views
- 1 replies
- 0 kudos
Not able to invoke model external model
I have followed below steps1) Created serving end point by for external model gpt-4-turbo and providing azure AI endpoint and key2) Now using langchain, i am trying to connect and invoke message from model in notebook model = ChatDatabricks(target_u...
- 5144 Views
- 1 replies
- 0 kudos
- 0 kudos
Based on your description, you are encountering a 500 Server Error when trying to use the Langchain ChatDatabricks integration with a Databricks Serving Endpoint connected to an external OpenAI GPT-4 Turbo model on Azure. This error usually indicates...
- 0 kudos
- 5714 Views
- 1 replies
- 0 kudos
Mlflow.evaluation fails to generate score
The execution of code stucks when evaluation of data start. eval_df = pd.DataFrame( { "inputs": [ "What is MLflow?", "What is Spark?", ], "ground_truth": [ "MLflow is an open-source platform f...
- 5714 Views
- 1 replies
- 0 kudos
- 0 kudos
The issue described—a Databricks notebook getting "stuck" during the evaluation phase using mlflow.evaluate—is most likely related to environment setup, model compatibility, or limitations with the mlflow.pyfunc.log_model and the evaluation utilities...
- 0 kudos
- 5760 Views
- 1 replies
- 0 kudos
I need a sample code or process which will help us to dynamically select the prompt template
We need a sample code or process which will help us to dynamically select the prompt template based on the prompt given as an input through the model legacy serving endpoint
- 5760 Views
- 1 replies
- 0 kudos
- 0 kudos
To dynamically select a prompt template in Databricks based on the input prompt received through a legacy model serving endpoint, you can implement a Python function that maps incoming prompts to specific templates. This often involves using conditio...
- 0 kudos
- 8087 Views
- 1 replies
- 0 kudos
How to perform combined search on structured and unstructured data in databrick using RAG or other
I created a RAG application in databricks which performs the following steps:1. Extract text from PDF files2. Prepare embeddings on extracted text and create vector search index3. Create a LLM model and served the model which can answer question base...
- 8087 Views
- 1 replies
- 0 kudos
- 0 kudos
You can achieve combined retrieval across both PDF-extracted unstructured data and multiple columns from structured Delta tables in Databricks, but there are important considerations and available patterns to optimize this workflow for your RAG appli...
- 0 kudos
