How to Build a Context Graph

Before you start

This guide assumes you have AI agents in production (or heading there) that make decisions affecting real users, revenue, or compliance. If your agents are single-turn assistants answering questions from a static knowledge base, you may not need a full context graph yet. A well-tuned RAG pipeline might be sufficient.

You need a context graph when your agents start exhibiting these symptoms: contradicting themselves across steps, applying expired rules, losing track of state after tool calls, or making decisions they cannot explain.

Before building anything, map every decision your agent makes. Not the prompts — the decisions. For each decision, document three things: what information feeds it, where that information comes from, and how often it changes.

Most teams discover that their agents make far more decisions than they realized. A customer service agent does not just "answer questions." It decides which policy applies, whether the customer qualifies for an exception, what the correct escalation path is, and whether previous interactions should influence the response.

Starting from your decision inventory, identify the core entities: users, products, rules, permissions, policies, configurations. Then map the relationships between them. This is your knowledge graph layer — the foundational structure that captures what exists and how things connect.

Keep the schema tight. A common mistake is modeling everything. You only need entities and relationships that influence agent decisions. If a piece of information never enters a decision path, it does not belong in the context graph.

This is where the context graph diverges from a standard knowledge graph. Every node and edge gets temporal metadata: when it becomes valid, when it expires, and how freshness is verified.

A pricing rule that was correct last quarter should not silently drive decisions this quarter. A user permission granted temporarily for a project should not persist indefinitely. Temporal validity makes time a first-class property of every piece of context, preventing the most common class of production agent failures: decisions based on stale information.

Not all context applies to all situations. A discount rule for enterprise customers should not activate for individual users. A compliance requirement for EU data should not apply to US-only operations. Applicability logic defines the conditions under which each piece of context is relevant.

This is the layer that distinguishes context graphs from retrieval systems. RAG answers "what is similar?" Applicability logic answers "what applies here?" — a fundamentally different question that requires structured conditions, not vector similarity.

Production systems run on exceptions. The standard refund policy says 30 days, but VIP customers get 90. The API rate limit is 100 requests per minute, but the analytics pipeline has an override. Exceptions are not bugs — they are the reality of business operations.

Model exceptions as first-class citizens in your context graph: who authorized the exception, when it expires, what it overrides, and under what conditions it applies. Without explicit exception modeling, agents either ignore exceptions (wrong behavior) or hallucinate them (dangerous behavior).

Every agent decision should produce a trace: what context was available, what was selected, what was excluded, what rules applied, and what the outcome was. This is not logging — logging records what happened. Traceability records why it happened.

Decision traces serve three purposes. They enable debugging when agents produce unexpected outputs. They provide the audit trail that regulated industries require. And they create the feedback loop that makes the context graph better over time — you can see exactly which context elements led to good and bad decisions.

The context graph is infrastructure. It needs an integration layer that sits between the graph and your agent runtime. At each decision point, the agent queries the context graph with its current state, and the graph returns the applicable, valid, structured context for that specific decision.

The integration pattern depends on your agent framework. Some teams use MCP (Model Context Protocol) servers as the query layer. Others build REST APIs or GraphQL endpoints. The implementation details matter less than the principle: the agent should never assemble its own context from raw sources. The context graph provides pre-validated, pre-structured, temporally-verified context at each step.

What comes after step seven

Building a context graph is not a one-time project. It is an evolving infrastructure that improves as you learn from production decisions. The decision traces from step six feed back into every other step — revealing missing entities, stale temporal rules, insufficient applicability conditions, and unmodeled exceptions.

The teams that get the most value from context graphs treat them like production databases: they have schemas, migrations, monitoring, and on-call processes. The context graph is not a nice-to-have. It is the decision infrastructure layer that determines whether your agents are reliable or merely plausible.

Continue exploring