Enterprise AI Has a Serious Context Problem

[Brahmareddy]

After coming back from the Databricks Data + AI Summit this year, one thought has stayed with me. I keep hearing Databricks co-founder and CEO Ali Ghodsi talk about something that I am also starting to see in my own Data + AI POCs.

For most enterprise AI use cases, the biggest problem may not be model intelligence anymore.

It is context.

Think about how much business knowledge exists inside a company.

Meetings where important decisions were made. Business rules that teams follow every day. Past decisions and exceptions. Workflows built over many years. And a lot of knowledge that simply lives inside employees’ heads.

AI does not automatically know any of this.

I have seen this while building POCs as well.

Today, connecting an LLM and building an AI agent is becoming easier. We can create an interesting demo surprisingly fast.

But then the real questions start.

Does the AI know what “revenue” actually means inside your company?

Does it know which customer definition is trusted?

Does it know which system is the source of truth?

Does it understand why a business process works the way it does?

That is where things become difficult.

The model may be very intelligent, but without the right business context, it can still give the wrong answer.

This is why I am becoming more interested in the context layer of enterprise AI.

Databricks is also moving in this direction with capabilities such as Genie Ontology, which is designed to bring business meaning and organizational context closer to data and AI.

Ali has also made another point that I think is important. Many organizations are still very early in actually using AI to automate work and create meaningful business value.

From what I am seeing, the next phase of enterprise AI may not be about finding an even smarter model.

It may be about helping AI understand how our businesses actually work.

AI already knows a lot about the world. Now we need to help it understand our organizations.

For data engineers, I think this creates an interesting opportunity.

We have spent years building pipelines that move data.

Now we may also need to help build the context that gives that data meaning.

What are you seeing in your Data + AI projects? Is context becoming a challenge for you too?

[juanlozadab]

Great post. I agree with the diagnosis, but I'm not sure the problem is new. I think agents are just making it visible.

We've always had this gap. It was the gap between a junior and a senior engineer. A junior can know the tools very well. But a senior also understands the business: what "revenue" actually means here, which customer table people trust, why a process has that strange exception. Knowing the tools gets the query to run. Knowing the business gets it to return the right answer.

The same gap also existed between roles. The data engineer knew the tools and the data. The business or functional analyst knew what it meant and how the business used it. Closing that gap took meetings, validations and a lot of back-and-forth between the two.

Now we expect an agent to do both jobs at once: understand the question like the analyst and build the answer like the engineer. It's very good at the second part. It knows the tools and syntax better than most of us. But it arrives with none of the business context, so it gives the classic junior answer: technically correct, and wrong for the business.

Before, that knowledge lived in people's heads and in conversations between roles. With agents it can't stay there. It has to be written down somewhere a machine can use: definitions, trusted tables, lineage, documented rules.

So for me the real shift isn't that context suddenly matters. It's that the knowledge that made seniors and analysts valuable now has to become an explicit, maintained asset. That's a big opportunity for data engineers who can work on both sides.