Building an AgenticLakehouse: Interacting with Databricks Workspace via LangGraph and MCP

vinaygazula
New Contributor II

This project, AgenticLakehouse, explores the cutting edge of "Agentic Data Analytics." I didn't just want a chatbot; I wanted a "living" interface for the Lakehouse. The result is a Multi-Agent System that intelligently orchestrates tasks, from querying Unity Catalog to browsing the web, deployed directly as a Databricks App.

To achieve this, I separated the system into a "Brain" and "Hands":
• The Brain (LangGraph): A router architecture that assesses user intent and dispatches tasks to specialist agents (e.g., a Databricks Agent, a Web Search Agent, etc,.)
• The Hands (MCP): A custom Model Context Protocol server that standardizes how these agents discover schemas, inspect lineage, and safely execute Spark SQL queries

Tech Stack Breakdown:
• Orchestration: LangGraph (Router + Specialist Agents)
• Protocol: Model Context Protocol (FastMCP)
• LLM Models: Groq
• UI/App: Gradio (Databricks Apps)
• Compute: Databricks Serverless SQL, Databricks App, Render
• Observability: LangSmith

In the Medium post, I walk through the full architecture, including how to optimize Unity Catalog metadata for LLM context windows and managing state across multiple agents.

Advika
Community Manager
Community Manager

Looks great, solid LangGraph + MCP setup on Databricks Apps. Thanks for sharing, @vinaygazula!