AI Agents Should Not Start Every Conversation From Zero

[Brahmareddy]

Databricks Agent Memory brings an interesting new direction to building AI agents that can learn from past interactions.

Recently, Databricks introduced Agent Memory, and I think this is an interesting step toward making AI agents more useful in everyday business applications.

The idea is simple.

An AI agent should not have to start every conversation from zero.

Think about how we work with people.

When we ask a colleague for help, we don't explain our entire background, preferences, business processes, and previous decisions every single time.

They already know some of that context from working with us.

But many AI agents still behave differently.

You ask a question today, explain your requirements, and get a useful answer. A week later, you ask something similar, and the agent may need the same explanation again.

It feels like working with someone who forgets everything after every meeting.

I have noticed this challenge while building my own Data + AI POCs.

Connecting an LLM, building tools, and creating an agent are becoming easier. But making that agent useful across multiple conversations is a different challenge.

For example, imagine an AI data analyst.

Today, you ask it to calculate revenue. You explain that your company excludes refunds and cancelled orders from its revenue calculation.

The agent understands and gives you the correct result.

Next week, you ask for revenue by region.

Should you have to explain the same business definition again?

Ideally, no.

The agent should be able to use the relevant context from the earlier interaction, while still following the latest business rules and data permissions.

That is where memory becomes valuable.

And it is not just about remembering user preferences.

An agent may also benefit from remembering how a similar problem was solved, which approach worked, and what information was useful.

For data engineers, this creates another interesting responsibility.

We already think about data quality, governance, lineage, and access controls.

Now we also need to think about the quality of the context an AI agent remembers.

Because remembering the wrong information can be just as problematic as having no memory at all.

An outdated business rule, an incorrect assumption, or information from another user should not silently influence future answers.

Good agent memory needs to be relevant, reliable, secure, and easy to correct.

I see this as part of a bigger shift in enterprise AI.

We started by connecting models to data. Then we gave agents tools to perform actions. Now we are working toward agents that can maintain useful context across interactions.

The goal is not simply to make AI remember more.

It is to help AI remember the right things.

For me, this is another reminder that the future of enterprise AI is not only about smarter models.

It is also about better context.

An intelligent agent can answer a question. An agent with the right memory can build on what it already knows.

Curious to hear from the Databricks Community.

How are you handling memory in your AI agents today? Are you storing conversation history, maintaining user preferences, or building your own memory layer?

[DoTA]

Worth knowing before anyone builds on it: the docs now mark the earlier managed memory store as deprecated and point new agents at the newer agent memory plus agent sessions, which Databricks runs on Lakebase for you. The self-managed Lakebase route is labelled legacy too, so check which page you are reading.

On your point about remembering the wrong thing, the part I would look at first is scope. The docs say to set it from the verified end-user identity in trusted application code, never let the model pick it, and reject requests with no user identity instead of falling back to a shared scope. Scope is also not access control. The app's service principal can read every scope, so a bug in the app code is how one user's memory ends up in another user's answer.

For stale business rules like your refunds example, I would keep the definition in the governed layer (a metric view or the instructions) and let memory hold preferences and past approaches only. That way a changed rule can't be overridden by something the agent remembered last month. I didn't see anything in the docs about editing or deleting individual entries, so I'd test that before promising users they can correct it.

[Brahmareddy]

Thanks, @DoTA DoTA. Really useful points, especially around memory scope and keeping business rules separate from agent memory.

I agree that trusted business definitions should stay in the governed layer, while memory should focus on user preferences and past interactions.

Your point about testing how memory can be corrected or deleted is important too. Definitely something worth validating before moving to production. Appreciate you adding these details!