- 30562 Views
- 18 replies
- 33 kudos
Databricks Announces the Industry’s First Generative AI Engineer Learning Pathway and Certification
Today, we are announcing the industry's first Generative AI Engineer learning pathway and certification to help ensure that data and AI practitioners have the resources to be successful with generative AI. At Databricks, we recognize that generative ...
- 30562 Views
- 18 replies
- 33 kudos
- 5729 Views
- 4 replies
- 0 kudos
Unable to log MLFlow run for LangChain chain while using databricks-langchain library
Whenever I try to log my run it throws me the following error: MlflowException: Failed to save runnable sequence: {'0': 'RunnableParallel<query,context> -- Failed to save runnable sequence: {\'context\': "RunnableSequence -- Failed to save runnable s...
- 5729 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @heramb13 , Your chain is being serialized by MLflow; during that process MLflow re-imports each runnable by its module path. The error shows MLflow is trying to import ChatDatabricks from the legacy module path “langchain_databricks”, which isn’t...
- 0 kudos
- 4106 Views
- 1 replies
- 0 kudos
Unable to add dependencies to mlflow.langchain.log_model
Hello,I'm doing this-with mlflow.start_run(run_name="run1"): logged_chain_info = mlflow.langchain.log_model( # lc_model=os.path.join(os.getcwd(), 'full_chain'), this doesnt work too lc_model='/Workspace/Users/{user_name}/exp/d...
- 4106 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @yashshingvi , Thanks for the details—this is a common gotcha with MLflow “models from code.” Why your imports fail code_paths are only added to sys.path when the model is loaded (for inference/serving), not while the driver is executing mlflow.la...
- 0 kudos
- 4138 Views
- 1 replies
- 2 kudos
AI/BI Genie - Real world experience?
I have been tasked with creating a NLP interface for management to our internal datalake that has numerous joined tables around customer information. I have tried the hello-world llamaindex, langchain sql/pandas agents with limited success (too many ...
- 4138 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi there Sam,I work for Databricks. The pace of innovation with AI/BI is wild (fast). Databricks itself just migrated off a leading BI platform and now exclusively runs its business on AI/BI (Blog). We also have customer testimonials on our website, ...
- 2 kudos
- 360 Views
- 2 replies
- 2 kudos
Resolved! Where are GPT-5 and Gemini in Azure Databricks?
Are GPT-5 and Gemini not available in Azure Databricks? According to Native OpenAI Models Now Generally Available on Databricks | Databricks Blog and Google’s Gemini Models on Databricks | Databricks Blog these are both GA with no mention of platfor...
- 360 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Rjdudley ,They're not available yet on Azure Databricks. Here's up-to-date list of all available mode right now:Databricks-hosted foundation models available in Foundation Model APIs - Azure Databricks | Microsoft LearnSo, we need some patience ...
- 2 kudos
- 2583 Views
- 5 replies
- 3 kudos
Using Genie in Multi-Agent Systems — SQL Query Missing from Output
Hi Community,I’ve implemented a multi-agent system using Genie in Databricks, following the guide on "Use Genie in multi-agent systems | Databricks Documentation". The system integrates well with vector indexes, and everything works fine during model...
- 2583 Views
- 5 replies
- 3 kudos
- 3 kudos
When using a custom Genie agent with the Databricks Genie REST API (GetMessage endpoint), extracting SQL queries and results works well as a standalone agent. However, integrating this custom agent into a multi-agent framework (especially when combin...
- 3 kudos
- 4492 Views
- 2 replies
- 0 kudos
DatabricksFunctionClient : PERMISSION_DENIED when creating the client
Hello, I'm currently working with Databricks features to create an AI agent system. In the documentation it's written that the library DatabricksFunctionClient is needed to register custom tools to UC. I followed the different steps from the explana...
- 4492 Views
- 2 replies
- 0 kudos
- 0 kudos
This error is most often due to missing permissions, workspace types, or cluster configuration settings necessary for AI agents and custom tools to access Unity Catalog (UC) functions using Spark Connect in Databricks. The main causes and solutions a...
- 0 kudos
- 4422 Views
- 3 replies
- 2 kudos
Langchain SQLDatabase not fetching column names from table in Unity Catalog
I am building a text-to-sql agent using Langchain API.I created a SQLDatabase using: db = SQLDatabase.from_databricks(catalog="`my-catalog-name`", schema="my_schema", host="...", api_token="...", warehouse_id="12345678")When I call db.get_table_info(...
- 4422 Views
- 3 replies
- 2 kudos
- 2 kudos
The issue happens because SQLDatabase.from_databricks doesn’t automatically fetch column metadata from Unity Catalog tables in some LangChain versions, leading to malformed SELECT statements. A workaround is to manually query the catalog like you did...
- 2 kudos
- 4410 Views
- 1 replies
- 1 kudos
Advanced RAG Retrieval (Reranking, Hierarchical, etc) in Databricks
Issue with current documentation:I wish to perform advanced RAG using langchain, in Databricks. In the documentation, they tell how to use the vector endpoint url, and index stored in catalogs. But I could not find any advanced RAG algos that are eas...
- 4410 Views
- 1 replies
- 1 kudos
- 1 kudos
Greetings @meetiasha , yes—there’s a gap between Databricks’ basic “vector endpoint + catalog index” examples and truly advanced RAG, so below is a step‑wise, LangChain‑first playbook you can run entirely on Databricks notebooks with local vector sto...
- 1 kudos
- 4165 Views
- 1 replies
- 1 kudos
GenAI Cookbook - how to add source documents to output and open pdf file on a page
Hello,I am implementing RAG solution as per Databricks cookbook. Review App is working, references are provided as text chunks.I need to build functionality to open pdf file on a specific page as a reference. Is there a way to change ReviewApp to ope...
- 4165 Views
- 1 replies
- 1 kudos
- 1 kudos
You want your RAG solution (based on Databricks Cookbook) to display PDF files at specific pages as references in your Review App, rather than plain text chunks. You also wish to retrieve source documents from your serving endpoint, but your current ...
- 1 kudos
- 4214 Views
- 1 replies
- 0 kudos
Invoke Azure AI Search Endpoint through Databricks environment
Hi Team,Is there a possibility to invoke Azure AI Search Vector DB endpoint(external) in Databricks environment based on input data in Databricks table.Scenario: Client-Specific documents are already embedded in Azure AI Search Vector DB. Is there an...
- 4214 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, it is possible to invoke an Azure AI Search Vector DB endpoint from within a Databricks environment—allowing you to leverage your existing Azure resource for client-specific document retrieval, without needing to create a new vector database in ...
- 0 kudos
- 5968 Views
- 6 replies
- 3 kudos
Resolved! How to integrate genie in with databricks apps
How to integrate genie in with databricks - data apps i want a chatbot that behaves like genie
- 5968 Views
- 6 replies
- 3 kudos
- 3 kudos
Thank you for the code. However, when user asks a follow up question, app responds with irrelevant data. The context or history is not preserved. How to handle this?
- 3 kudos
- 1066 Views
- 1 replies
- 0 kudos
Deploying HuggingFace LLM model with MLflow task llm/v1/chat into Databricks
Hello,I am currently trying to deploy a HuggingFace LLM model to Databricks with the MLflow task llm/v1/chat in order to use it as a chat.I have tried several models like:TinyLlama/TinyLlama_v1.1 · Hugging FaceBSC-LT/salamandra-7b-instruct · Hugging ...
- 1066 Views
- 1 replies
- 0 kudos
- 0 kudos
Deploying HuggingFace LLM models to Databricks using MLflow’s llm/v1/chat task sometimes results in unexpected chat behaviors, usually due to prompt/template mismatches, model configuration issues, or pipeline setup requirements. Here’s a direct answ...
- 0 kudos
- 4059 Views
- 1 replies
- 0 kudos
Issue in creating endpoint for quantized gguf model with llama-cpp-python
Hello, Databricks Community,I am experiencing an issue while trying to serve a quantized model in gguf format using Databricks serving with the llama-cpp-python library.The model is registered using MLflow and pyfunc on Unity. The model loads without...
- 4059 Views
- 1 replies
- 0 kudos
- 0 kudos
The error code 132 typically means an illegal instruction was encountered, often caused by a CPU incompatibility with the code being executed—especially with libraries that use SIMD or hardware acceleration (e.g., llama-cpp-python, which is often com...
- 0 kudos
- 4059 Views
- 1 replies
- 0 kudos
Not able to invoke model external model
I have followed below steps1) Created serving end point by for external model gpt-4-turbo and providing azure AI endpoint and key2) Now using langchain, i am trying to connect and invoke message from model in notebook model = ChatDatabricks(target_u...
- 4059 Views
- 1 replies
- 0 kudos
- 0 kudos
Based on your description, you are encountering a 500 Server Error when trying to use the Langchain ChatDatabricks integration with a Databricks Serving Endpoint connected to an external OpenAI GPT-4 Turbo model on Azure. This error usually indicates...
- 0 kudos
- 4439 Views
- 1 replies
- 0 kudos
Mlflow.evaluation fails to generate score
The execution of code stucks when evaluation of data start. eval_df = pd.DataFrame( { "inputs": [ "What is MLflow?", "What is Spark?", ], "ground_truth": [ "MLflow is an open-source platform f...
- 4439 Views
- 1 replies
- 0 kudos
- 0 kudos
The issue described—a Databricks notebook getting "stuck" during the evaluation phase using mlflow.evaluate—is most likely related to environment setup, model compatibility, or limitations with the mlflow.pyfunc.log_model and the evaluation utilities...
- 0 kudos
