High Quality Data is no longer enough for AI based decisions. Governance, Lineage and Observability helped creation of Trusted Data Products to allow consumers to use the trusted information delivered by the Data Platforms.
Generative AI and Agentic AI introduces a different challenge. An AI agent does not consume data the same way as a traditional dashboard, report or analytical model. It needs to assemble information, understand relationships, interpret business meaning, consider history and current state, apply relevant policies and determine which evidence matters for the task it is performing. It means that even when the underlying Data is of High Quality, the context provided to the agent can still be poor. Data Engineering is evolving from Data Quality to Context Quality.
Data Quality Foundation
Enterprise data products needs to be trustworthy. A customer table with duplicate customers is not a trusted data product. A transaction table with missing records cannot reliably support financial reporting. A pipeline that is several hours behind when a decision requires current information has a freshness challenge. Data quality became a fundamental quality discipline. Frameworks checked whether data was accurate, complete, consistent, valid, unique and fresh. It used Monitoring, Data Contracts, Validation Rules, Observability and Governance on the data platform to make trusted data.
It created a stronger foundation for enterprise analytics and decision making. However, there is an implicit assumption in traditional Data Quality โ If the underlying data is trustworthy, the consumer can make a trustworthy decision from it. That assumption is not true for AI agents.
High Quality Data Product can still create Poor Context
Consider the fraud investigation case โ The bank has a high-quality Customer data product. Transaction data is accurate. Merchant information is governed. Device information is available and fresh. Historical fraud cases are maintained. Fraud policies are documented.
The bank has done everything normally expected from a mature Data Engineering organization. Now an agent receives the task - Investigate the transaction and determine whether it should be escalated for fraud investigation. The agent may receive the transaction correctly. But what if it does not receive the customerโs recent behavioral pattern? What if it receives the customer but misses the relationship between the device and other accounts? What if it retrieves a fraud policy but uses an older version that was not applicable when the transaction occurred? What if the transaction is current but the supporting fraud signal is several hours old?
Agent needs to know which customer is connected to the account. It needs to understand which device was used for the transaction. It may need the sequence of recent transactions rather than isolated transactions. None of these necessarily represent a traditional Data Quality failure. The underlying datasets may all pass their quality checks. The failure is in the context assembled for the decision. It leads to an important distinction โ Data Quality confirms whether the information can be trusted. Context Quality confirms whether the information is fit for the decision.
The individual data products may be correct. The assembled context can still be incomplete, contradictory, stale, poorly connected or insufficient for the decision. It is the Context Quality Gap.
Context Quality
Context Quality does not replace Data Quality. It builds on it. If the underlying data is inaccurate, context will obviously be unreliable. But even perfect data can become poor context when the wrong information is selected, important relationships are lost, business meaning is ambiguous, historical and current information are mixed incorrectly or critical evidence is missing. The quality of context is determined not only by the quality of its individual components, but also by how those components are assembled for a particular task.
It answers โ Is this the right information, connected correctly, meaningful for this task and sufficient for this decision?
Context Quality is Decision Specific
Data Quality can be evaluated at the dataset, table or column level. Context Quality needs to be evaluated in relation to a decision or task. For the fraud agent, the question is not โ Is the Customer table complete?. It is Does the agent have enough trusted information about this customer to investigate this transaction?
The question is not Are transactions fresh?. It is Is the transaction history available at the freshness required to identify the fraud pattern?. The same data can be perfectly suitable context for one decision and completely insufficient for another.
Context Quality Dimensions
Relevance is giving the agent appropriate information that is relevant to the decision it is trying to make. There may be thousands of tables, documents, APIs and definitions, but more information does not automatically mean better reasoning. For a fraud investigation case, recent transactions, device relationships and fraud signals may be important while unrelated customer interactions adds noise. The goal is to provide right context for the decision.
Sufficiency is about whether the agent has enough information to make the decision with confidence. The agent does not need everything available in the org, but it should not be missing information that could materially change the outcome. In a fraud investigation, recent transaction behaviour, device relationships and the applicable fraud policy may all be necessary. Context can therefore be relevant and accurate, but still not be enough for the decision.
Semantic Integrity is related to preserving the meaning of the information as it moves across different data sources and into the agentโs context. An agent needs to understand not only individual records, but what those records represent and how they relate to each other. A transaction belongs to an account, an account belongs to a customer and a device can be associated with multiple accounts. If those meanings or relationships are lost, the data may still be correct while the context becomes misleading.
Timeliness is on whether the information reflects the right state at the right point in time. A fraud agent may receive an accurate transaction, but an outdated fraud signal or a policy that was not applicable when the transaction occurred can change the decision. This makes timeliness different from simply checking whether data is fresh. The real query is whether the information is timely for the decision being made.
Consistency becomes important when an agent brings information together from multiple sources. Each source may be individually correct, but the combined context can still contain conflicting values, definitions or business rules. If one system identifies an account as active while another provides a conflicting status, the agent needs to know which information should be trusted. Context Quality requires the assembled information to stitch a consistent story.
Provenance is knowing where the information in the agentโs context came from and what evidence supports it. When an agent makes an important recommendation, simply presenting the answer is not enough - the organization needs to understand which data, document, policy or signal influenced that recommendation. Provenance connects the agentโs decision back to the underlying evidence and gives Data Engineering a way to trace and validate the context used.
Policy Authorization is making sure that the information available to an agent is information it is actually allowed to use. There may be highly relevant and trustworthy data, but that does not mean every agent should have access to it for every task. As context is assembled across multiple sources, the original access controls and data policies need to remain effective.
Data Quality to Context Quality checks
Traditional Data Quality checks validate โ Is the customer identifier valid? Are transactions complete? Are required fields populated? Is the data fresh? and are duplicate records present?
Context Quality checks evaluate different aspects โ Does the context contain the information required for this decision? Are the relevant entities correctly connected? Are definitions consistent across the information being used? Is the information temporally valid? Is the agent authorized to access the information? Is the context sufficient without unnecessary information?
These checks become part of the data platform development lifecycle.
Agent becomes a new quality signal
In traditional Data Engineering, data quality monitoring shows where the data platform has problems. In an agentic world, agent behavior can show where the data platform has context problems.
Imagine the fraud agent repeatedly asks for a missing customer relationship. That may indicate a context assembly problem. Suppose it consistently confuses two business definitions. That may indicate a semantic problem. Suppose it makes different recommendations because historical information is unavailable. That may indicate a temporal context problem. Suppose it cannot explain the evidence behind a recommendation. That may indicate a provenance problem. The agent therefore becomes another feedback mechanism.
Context Quality evolves with the Agent
Just as the Context Layer should be built around a decision, Context Quality evolves through the same feedback loop. The feedback should flow back into the data products, semantic layer, governance model and context assembly process. This creates an change in the Data Engineering lifecycle. Agent behavior becomes a signal that can tell DE what information the platform needs to improve.
What changes for Data Engineering?
Data Engineers increasingly need to understand how those capabilities combine when information is consumed by agentic systems. Semantic models provide meaning. Governance provides control. Context Layer brings these capabilities together for a task. Context Quality provides a way to ask whether the resulting information environment is fit for the decision.
It is a significant expansion of the Data Engineer role. Data Quality makes data products trustworthy. Context Quality helps it become decision ready. The goal is to build context that an intelligent system can trust when it matters.
