CCA-F Study Day 26/20: Context Engineering & Compaction — Long-Horizon Agent Reliability
📚 CCA-F Study Day 26/20: Context Engineering & Compaction
Review Cycle Day 6 — Long-Horizon Agent Reliability & Context Management Architecture
Cross-Domain Focus: Domain 5 (Context Management & Reliability, ~15%) + Domain 1 (Agentic Architecture & Orchestration, ~25%)
📌 Today's Focus
Today we tackle one of the exam's most nuanced topics: context engineering — the art of curating what goes into the finite context window to maximize agent reliability over long-horizon tasks. This is where Domain 5 and Domain 1 collide: context management strategies directly determine whether your multi-agent architectures succeed or fail at scale.
The exam loves testing the tension between "more context = more information" and "more context = worse performance." You need to know when to add context, when to prune it, and exactly which mechanisms (compaction, tool-result clearing, note-taking) to apply.
🧠 Core Concepts
1. Context Rot — The Fundamental Problem
Context rot is the degradation of model performance as the context window fills up. Despite supporting 200K+ tokens, Claude's ability to accurately recall and reason over information decreases as token count grows. This is due to:
- Attention budget depletion — Transformers create n² pairwise relationships for n tokens. More tokens = thinner attention spread.
- Training distribution bias — Models trained more on shorter sequences have fewer specialized parameters for long-range dependencies.
- Performance gradient — Not a cliff, but a slope: models remain capable at longer contexts but show reduced precision for retrieval and reasoning.
Exam implication: The correct answer is NEVER "just use a bigger context window." The exam tests whether you understand that context is a finite resource with diminishing marginal returns.
2. Context Engineering vs. Prompt Engineering
Anthropic explicitly distinguishes these:
- Prompt engineering = writing and organizing LLM instructions (system prompts, few-shot examples)
- Context engineering = curating and maintaining the optimal set of tokens during inference, including ALL information that lands in context (system prompts, tools, MCP data, message history, tool results)
Context engineering is iterative — the curation happens each time you decide what to pass to the model in a multi-turn agentic loop.
3. Compaction — The Primary Long-Horizon Mechanism
Compaction is the practice of summarizing a conversation nearing the context limit and reinitiating with that summary. In the Agent SDK:
# The Agent SDK handles compaction automatically via SystemMessage
# You'll see a SystemMessage with subtype "compact_boundary"
# when compaction occurs
# In Claude Code, compaction fires via the PreCompact hook:
hooks = {
"PreCompact": [archive_full_transcript_hook] # Save before compaction
}
# API-level compaction (Messages API):
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=8096,
system="...",
messages=messages,
# When context approaches limit, implement:
# 1. Detect token count approaching threshold
# 2. Summarize conversation so far
# 3. Start fresh context with summary + current task
)
5 Risks of Compaction (Progressive Summarization):
- Information loss — Details dropped during summary
- Bias amplification — Summaries emphasize what model finds "interesting"
- Error propagation — Mistakes in early turns persist in summaries
- Temporal confusion — "When did something happen?" gets lost
- Authority dilution — Source attribution lost ("the docs say" becomes "it was mentioned")
4. Tool-Result Clearing — The Second Lever
Tool results (especially from search, file reads, or API calls) are often the biggest source of context bloat. The pattern:
# After the agent has processed a tool result and extracted
# what it needs, clear the raw result from context:
def clear_old_tool_results(messages, max_age_turns=5):
"""Replace old tool results with summaries to save context."""
for i, msg in enumerate(messages):
if msg["role"] == "user" and "tool_result" in str(msg.get("content", "")):
turns_ago = (len(messages) - i) // 2
if turns_ago > max_age_turns:
# Replace verbose result with compact summary
msg["content"] = [{"type": "tool_result",
"tool_use_id": original_id,
"content": "[Cleared - summary: found 3 matching files]"}]
return messages
5. Just-in-Time Context Retrieval
Instead of pre-loading all possible context, maintain lightweight identifiers and load data on demand:
- File paths — Don't read entire files into context; use Glob/Grep to find, then Read only what's needed
- Stored queries — Save SQL/API queries, re-execute when needed rather than caching results
- Progressive disclosure — Let agents discover context layer by layer through exploration
Claude Code's hybrid model: CLAUDE.md files loaded up-front (stable context) + glob/grep for just-in-time retrieval (dynamic context). This is the exam-correct pattern.
6. System Prompt "Altitude" — The Goldilocks Zone
Anthropic identifies two failure modes for system prompts:
| ❌ Too Low (Brittle) | ❌ Too High (Vague) | ✅ Right Altitude |
|---|---|---|
| Hardcoded if-else logic in prompts | Vague high-level guidance | Strong heuristics + concrete signals |
| "If user says X, respond with Y" | "Be helpful and professional" | "When handling refund requests, verify order status first, then check policy" |
| Fragile, high maintenance | No actionable guidance | Specific enough to guide, flexible enough to generalize |
🚫 Anti-Patterns & Exam Traps
| ❌ Wrong Answer (Exam Trap) | ✅ Correct Approach | Why It's Wrong |
|---|---|---|
| "Use a larger context window to fit everything" | Curate minimal high-signal context; compact when needed | More tokens = worse attention focus = context rot |
| "Pre-load all relevant documents at the start" | Use just-in-time retrieval with tools | Wastes context budget on potentially irrelevant data |
| "The agent will remember everything from earlier turns" | After compaction, only the summary remains; verify critical info is preserved | Compaction loses details — can't reference what's been summarized away |
| "Use a single long-running session for the entire workflow" | Use multi-agent architecture with context isolation | Single session accumulates context rot; isolated agents start fresh |
| "Add more instructions to the system prompt to cover edge cases" | Use canonical few-shot examples instead of exhaustive rules | Laundry lists dilute attention; examples are "pictures worth 1000 words" |
| "Tools don't affect context budget" | Tool definitions AND results consume context — design token-efficient tools | Every tool definition + every result returned adds to context load |
💻 Code Examples
Pattern 1: Context-Aware Agent Loop with Compaction Detection
import anthropic
client = anthropic.Anthropic()
def run_agent_with_context_management(task: str, tools: list, max_context_tokens: int = 180000):
messages = [{"role": "user", "content": task}]
system_prompt = "You are a research agent. Use tools to gather information."
while True:
# Estimate current context size
estimated_tokens = estimate_token_count(system_prompt, messages, tools)
# Compact if approaching limit (80% threshold)
if estimated_tokens > max_context_tokens * 0.8:
summary = compact_conversation(messages)
messages = [
{"role": "user", "content": f"[Context Summary]\n{summary}\n\n[Continue Task]\n{task}"}
]
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=4096,
system=system_prompt,
tools=tools,
messages=messages
)
if response.stop_reason == "end_turn":
return response.content[0].text
# Process tool calls
messages.append({"role": "assistant", "content": response.content})
tool_results = execute_tools(response.content)
messages.append({"role": "user", "content": tool_results})
# Clear old tool results to prevent bloat
messages = clear_stale_tool_results(messages, keep_last_n=3)
def compact_conversation(messages: list) -> str:
"""Use Claude to summarize the conversation so far."""
summary_response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=2048,
messages=[{
"role": "user",
"content": f"""Summarize this conversation concisely, preserving:
1. Key findings and decisions made
2. Current task status and next steps
3. Any critical constraints or requirements discovered
4. File paths, IDs, or identifiers that may be needed later
Conversation:
{format_messages(messages)}"""
}]
)
return summary_response.content[0].text
Pattern 2: Multi-Agent Context Isolation for Long Tasks
# Instead of one agent accumulating context, distribute work:
async def research_pipeline(topic: str):
# Agent 1: Gather sources (fresh context)
sources = await run_agent(
task=f"Find the top 5 most relevant sources on: {topic}",
tools=[web_search_tool, url_fetch_tool],
max_turns=10
)
# Agent 2: Analyze each source (fresh context per source)
analyses = []
for source in sources:
analysis = await run_agent(
task=f"Analyze this source and extract key findings:\n{source}",
tools=[read_tool],
max_turns=5
)
analyses.append(analysis)
# Agent 3: Synthesize (fresh context, only receives summaries)
synthesis = await run_agent(
task=f"Synthesize these research findings into a report:\n{analyses}",
tools=[write_tool],
max_turns=5
)
return synthesis
# Each agent gets a FRESH context window — no context rot accumulation!
# The pipeline pattern passes artifacts, not conversation history.
📖 Reading (20 min)
- Effective Context Engineering for AI Agents — Anthropic's definitive engineering post on this topic. Read the entire thing.
- Effective Harnesses for Long-Running Agents — How Claude Code manages compaction for unlimited session length.
- Long Context Prompting Tips — Official documentation on placing documents at the top, using XML structure.
🛠️ Hands-On Exercise (20 min)
Build a Context-Managed Research Agent:
- Create a simple agent loop that calls a "search" tool repeatedly (mock it to return 500-token results each time).
- Implement a token counter that estimates context size after each turn.
- At 80% capacity, trigger compaction: summarize the conversation so far and restart with the summary.
- Add tool-result clearing: after 3 turns, replace old tool results with one-line summaries.
- Run the agent for 20 simulated turns and verify it doesn't degrade.
Bonus: Add a PreCompact hook that saves the full transcript to a file before compaction fires.
📝 Quick Quiz
Q1: An agent has been running for 45 turns and is approaching the context window limit. Which approach best maintains performance?
A) Increase max_tokens to extend the context window B) Compact the conversation by summarizing earlier turns and restarting with the summary C) Switch to a model with a larger context window D) Store all previous turns in a database and load them back when needed
Q2: A multi-agent research system is experiencing degraded output quality after processing many documents. What is the MOST LIKELY root cause?
A) The model is running out of GPU memory B) Context rot — accumulated tool results and conversation history have diluted the agent's attention C) The documents contain conflicting information D) The system prompt is too short
Q3: An architect is designing a system where Claude needs to analyze 50 log files. Which approach follows context engineering best practices?
A) Load all 50 files into context at once for comprehensive analysis B) Use just-in-time retrieval: have the agent use Glob to find relevant files, then Read only the files needed for the current sub-task C) Summarize all files in a preprocessing step and pass the summaries to Claude D) Split the 50 files across 50 separate API calls with no coordination
Answers: Q1: B — Compaction is the primary mechanism for long-horizon context management. A increases output tokens (not context), C doesn't solve rot, D reintroduces the same bloat. Q2: B — Context rot from accumulated tool results is the classic failure mode. GPU memory is infrastructure (not model behavior), conflicting info wouldn't cause quality degradation across the board. Q3: B — Just-in-time retrieval (Claude Code's hybrid model). A causes context rot, C loses details, D has no coordination.
👀 Tomorrow's Preview
Day 27 will tackle Extended Thinking & Budget Tokens — how thinking budget affects agent reasoning depth, when to enable extended thinking, and how it interacts with context management (thinking tokens don't count against output but DO affect latency and cost).