New Agentic AI Ecosystem in Databricks

[Bitrip007]

it seems Databricks is moving from traditional RAG pipelines toward agent-native architectures with Lakebase Search, Genie Agents, MCP integration, and semantic business context.

For teams building enterprise GenAI applications today:

• Are you still building custom RAG pipelines using Vector Search + LangGraph?
• Or are you planning to adopt the new Databricks agent ecosystem?
• Which components would you still build yourself, and which would you rely on Databricks to manage?

Curious to hear how enterprise architects are thinking about this shift.

[DoTA]

Given Databricks is now very mature in the Agentic/AI landscape comparing to 1 year ago, we are starting to leverage most of the capabilities from Databricks such as:

1. Vector Search for Vector Database -> Transitioning to Lakebase Search.

2. Lakebase for Agents Memory -> Transitioning to UC Managed Memory.

3. MLFlow for Agentic Lifecycle Management and Evaluation.

But there are still some components we intend to manage our self:

1. Graph Capabilities, Databricks simply does not have this capability, GraphFrame can only support Graph Analytics but not full on Agentic graph usage such as Knowledge Graph or Graph Traversal, ... -> Would love to see Lakebase support AGE.

2. Given the change in Genie PAYG model, we intend to own our Text to SQL capability to save cost, as this thing is getting more and more expensive.