Context window budgeting under token pressure

You have 128k tokens available and a monorepo with 2,400 TypeScript files. The agent needs to refactor a cross-cutting interface used in 47 modules. Do you send the full call graph? Summaries of each file? Just the interface definition and hope the LLM infers correctly?

This isn't hypothetical. Token budget exhaustion is the #1 silent failure mode in production AI coding tools. Unlike compilation errors that fail loudly, running out of context mid-task produces subtle degradation: the agent "forgets" earlier constraints, generates code that conflicts with files it saw 80k tokens ago, or silently drops test coverage because the examples scrolled out of the window.

Degradation starts before you hit the limit

Most engineers think context window problems happen when you exceed the model's maximum tokens. In practice, quality drops much earlier. A 128k context window shows noticeable degradation once you pass 60–70% fill. The model still runs, but it starts missing dependencies between early and late sections.

We've observed three distinct phases as token pressure increases:

**Phase 1 (40–60% fill):** Coherent but narrowing scope. The model focuses on recent context and ignores older material unless explicitly reminded. It re-asks questions you answered two exchanges ago.

**Phase 2 (60–80% fill):** Selective amnesia. It forgets specific files, function signatures, or constraints mentioned earlier. The plan stays coherent, but execution drifts from requirements.

**Phase 3 (80%+ fill):** Context thrashing. The model oscillates between contradictory states, sometimes hallucinating that it already made changes it hasn't touched, or re-implementing solutions you explicitly rejected.

The percentage thresholds vary by model family and task type, but the pattern holds: degradation is gradual, not binary. Budget for 60–70% utilization in normal operations to leave headroom for complex tasks.

The iron triangle: coverage vs. fidelity vs. latency

Every context window decision lives at the intersection of three forces:

**Coverage** is how much of the codebase you include. Sending 200 related files gives the agent a complete picture but consumes your budget fast. Narrow context (just the 5 files you think matter) runs cheap but the agent can't see the dependency that breaks your build.

**Fidelity** is how much detail you preserve per item. Full file contents are expensive but unambiguous. Summaries like "database client with connection pooling" save tokens but lose the actual connection string format your validation logic depends on.

**Latency** is the time cost of large prompts. A 100k token context takes 8–12 seconds just to process before the LLM starts generating. Tight iteration loops—plan, edit, compile, retry—demand faster feedback than "wait 10 seconds to see if the agent remembered your constraint."

You cannot optimize all three. Choosing where to compromise defines your system's behavior under pressure.

Tiered context: send what's essential, summarize the rest

The practical middle ground is **tiered inclusion**. Not every file needs the same fidelity.

Start with the **anchor files**—the 3–5 modules the agent will directly edit. Send these in full, with line numbers and current state. This is your ~15–25k token baseline.

Next, identify **direct dependencies**: imports the anchor files use, interfaces they implement, test utilities they call. For these, send a **signature view**: public API surface, type definitions, docstrings. Skip private helper functions and implementation details. A 400-line class compresses to a 40-line signature consuming ~300 tokens instead of 3,000.

Finally, add **ambient context**: related files that provide conceptual grounding but won't be edited. Use structured summaries:

File: src/db/migrations/0042_add_audit_logs.sql

Purpose: Adds audit_logs table with user_id, action, timestamp

Constraints: user_id must reference users.id, action is ENUM type

Modified: 2024-11-03

This gives the agent enough to avoid breaking invariants (don't generate code that assumes audit_logs.action is a string) without burning 5k tokens on the full migration DDL.

Load context in priority tiers with backpressure signals. Tier 1 is immovable: current instruction, immediate edit targets, active constraints. Tier 2 includes recent conversation turns and directly referenced files. Tier 3 covers broader project context. Tier 4 is nice-to-have background.

At 70% capacity, Tier 4 gets dropped. At 85%, Tier 3 gets compressed. At 95%, you're down to Tier 1 and heavily summarized Tier 2. This creates predictable degradation: the agent becomes narrower in scope but doesn't hallucinate missing information or silently violate constraints it can no longer see.

When fidelity beats coverage: the shared interface pattern

Assume you're changing a `Logger` interface used in 120 files. **High-coverage / low-fidelity** would send one-line summaries of all 120. **Low-coverage / high-fidelity** sends the full implementation of the 5 most complex consumers.

In practice, the latter wins. Why? Because the failure mode for underspecified changes is *silent divergence*. If you tell the LLM "update Logger calls in 120 files" with only summaries, it will generate syntactically valid code that compiles but misses semantic nuances—like that `logger.error()` in the payment service must include a `transaction_id` field for audit compliance, while other services don't have that requirement.

Send fewer files with full context. The agent generates correct implementations for those 5, you extract the **pattern** from the diffs, then apply that pattern mechanically (via AST transforms or another agent pass) to the remaining 115. This is the "anchor and extrapolate" strategy: high confidence on a subset, then systematic rollout instead of 120 simultaneous low-confidence guesses.

Dynamic pruning when you hit the ceiling

Even with tiering, you'll hit limits. A refactor touching 60 files exceeds any reasonable budget. The naive solution—truncate the oldest context—creates the "amnesia bug" where the agent's 50th edit contradicts guidance from the 5th.

**Compression on collision** is more robust. When token pressure forces a choice, replace entire blocks with *executable facts*:

• "All database clients in `src/services/*` follow the pattern: instantiate with `getConnection()`, wrap queries in `withTransaction()`, call `.close()` in finally blocks."
• "Test files under `tests/integration/` require environment variables `TEST_DB_URL` and `AWS_LOCALSTACK_ENDPOINT`."

These compressed rules are **checkable**. The agent can generate code, the validation loop can verify the pattern is present, and retry with the specific rule re-emphasized if violated. Contrast this with dropping the files entirely and hoping the LLM "just knows" your conventions.

In Goatfied's architecture, every loop iteration re-plans context budget: after the compile step, we measure what tokens the error messages and compiler output consumed, then adjust the next retry's context to preserve room for diagnostic data. If you sent 80k tokens in round one and got back 15k of error spans, round two's prompt drops to 65k of code context so the full error details fit.

After three or four retries, conversation history alone can consume tens of thousands of tokens. Each retry includes the previous error, the attempted fix, and the new error. Set a maximum retry depth—typically three retries per edit. Before you hit that limit, truncate older conversation turns from context. Keep the most recent error and current file state, but drop the first retry's full trace.

Reserve budget for validation artifacts

The hidden token sink is validation feedback. A compile error across 12 files generates 8k tokens of output. Test failures with stack traces can hit 20k. If your initial context consumed 100k, you've now exceeded a 128k window before the agent sees the errors.

**Reserve 30% for feedback loops.** If your model's window is 128k, treat your code context budget as 90k. Plan prompts should stay under 60k so you have room for plan output, file proposals, and revision feedback without truncation.

In Goatfied, we enforce this with a **token budget contract** at the start of each agent loop. The planner receives a remaining-token count and must allocate: X for code context, Y for proposed changes, Z for validation results. If the planner exceeds X, the loop rejects the plan before calling the LLM, forcing re-scoping rather than silently dropping context mid-stream.

For edits that must touch large files, stream validation feedback incrementally rather than loading everything at once. Run the compiler first. If it fails, feed only the compiler error back to the agent. Don't append the full lint report and test output until the code compiles.

This keeps the validation step's token count proportional to the severity of the error. A single type mismatch might cost 2k tokens to describe; a clean compile followed by one failing test might add 8k. But you avoid dumping 50k tokens of cascading errors from a file that doesn't even parse.

Goatfied's validate-then-retry loop implements this by gating each feedback type. Compile errors block lint runs. Lint failures block test runs. Each gate is a checkpoint where the agent can consume feedback and retry without accumulating the full error stack.

Canonical failure patterns and detection

**The model edits a file that doesn't exist.** This is the canonical token pressure bug. Your agent loop processes turn 8 of a multi-file refactor. The model confidently generates an edit block for `src/utils/validation.ts`—a file that was in context during turn 2 but got evicted to make room for newer conversation. The file path is real, the function name matches your conventions, but the actual file was never opened or was dropped three turns back.

Detection: Track which files are actually in the current context window. If an edit references a file not in the tracked set, reject it before attempting to apply. Goatfied's constraint phase catches this—every planned edit gets checked against the known file tree and current context snapshot before execution.

**The model re-implements what it just did.** Token pressure causes temporal confusion. The model loses track of its own recent actions and re-applies changes, sometimes with slight variations that conflict with the first version. This typically happens when the confirmation of successful edits gets pushed out of context.

Mitigation: After each successful edit, inject a brief confirmation into context that costs minimal tokens: `✓ [turn 4] Added error handling to processRequest (src/api/handler.ts:45-52)`. This breadcrumb trail helps the model track completed work without keeping full file history in context.

**Constraint violations after the third or fourth iteration.** You've set explicit boundaries: "Don't touch the database schema" or "Keep all changes in the `src/experimental/` directory." The first two edits respect these constraints. By edit four or five, the model is proposing a migration file or modifying a core module.

This isn't the model being rebellious—it's forgotten the constraints. Instructions given in the system prompt or turn 1 have effectively zero influence once token budget is saturated with code, diffs, and conversation.

Goatfied's agent loop treats constraints as a persistent phase, not a one-time check. Before each edit, the constrain step re-validates against the rule set. This costs tokens—maybe 200–400 per turn—but prevents expensive rollbacks when the model violates boundaries you set 30k tokens ago.

Instrument actual usage in production

Track token consumption in real-time. If you're at 70% of your model's window, you're already in the danger zone. Most failure patterns become visible before you hit hard limits.

For each LLM call, record the input token count (prompt + context) and output token count (generated code/plan). Track this per-step in the agent loop: planning, editing, validating. Log these per-PR or per-task. Export to a time-series database or structured logs so you can query percentiles.

Track **tokens per successful edit**. If you consistently burn 80k tokens to fix a 20-line function, your context strategy is mis-targeted. Compare across task types: renaming a function should cost <10k; adding a feature with new endpoints might justify 60k.

Second, measure **context-change correlation**. When a retry succeeds after initial failure, diff the token allocation between attempts. If you dropped 15k of unrelated test mocks and kept 5k of the actual interface definition, that's signal. Build heuristics: "For interface refactors, prioritize callers in `src/` over `tests/`, 3:1 ratio."

We've found that 60–75% utilization indicates healthy operation with room to adapt. 75–90% suggests you're occasionally making hard tradeoffs but still functioning. Above 90% means you're regularly in degraded mode and should rethink your approach to that class of task—maybe it needs to be decomposed differently, or maybe certain context simply isn't as valuable as you thought.

Optimize prompt structure

How you arrange context matters as much as what you include.

**Front-load critical sections.** Place the code under active edit at the top of your context block, followed by immediate dependencies, then peripheral files. When vendors truncate, they usually drop tail content first. Ensure the most relevant material survives.

**Symbolic references over full inclusion.** For large dependency trees, pass function signatures and type definitions rather than full implementations. `def process_payment(cart: Cart, token: str) -> PaymentResult` conveys the interface without burning 500 tokens on internal payment gateway logic the agent doesn't need to modify.

**Collapse repetitive patterns.** If your codebase has twenty nearly-identical CRUD handlers, include one full example and stub the rest with comments like `# UserHandler: follows same pattern as ItemHandler above`. The agent can extrapolate structure without seeing every repetition spelled out.

For code context, strip comments in files you're not actively editing (you already parsed them during planning). Remove import blocks except for the specific imports relevant to your current edit target. Collapse unchanged function bodies to signatures. A 3,000-token utility file often compresses to 400 tokens of interface surface area without losing the information needed to make correct edits.

Checkpoint-based resets with continuity

When you genuinely can't fit everything needed into one context window, implement checkpoint-based resets. At natural boundaries—after successful compilation, after a test suite passes, after merging one logical unit of work—serialize the critical state into a compact checkpoint.

A checkpoint might include: the current file state (diffs only, not full files), active constraints, recent decisions with rationale, and validation results. This entire package often fits in 5–10k tokens. You then start a fresh context window with the checkpoint loaded, giving you full budget for the next operation while maintaining continuity.

Goatfied's small, reversible diffs make this particularly effective. Instead of checkpointing entire file contents, you checkpoint the sequence of diffs and their validation results. Rolling forward through checkpoints becomes replay of a compact operation log rather than restoration of bloated state.

Token budgeting is correctness, not optimization

Staying under token pressure isn't about squeezing more performance out of your AI workflow. It's about preventing a specific class of silent failures that corrupt multi-step tasks. Every pattern in this guide comes from debugging real agent loops that appeared to work but produced subtly broken output.

The goal isn't to never hit budget pressure. The goal is to hit it predictably, respond systematically, and maintain correctness throughout. If your AI coding assistant "gets confused" after a few back-and-forth exchanges, that's not vague UX friction—it's a measurable resource exhaustion problem with engineering solutions.

Related reading

• [Agentic planning for cross-package refactors](/blog/agentic-planning-for-cross-package-refactors)
• [Branch-aware context retrieval pipelines](/blog/branch-aware-context-retrieval-pipelines)
• [Completion ranking with repository signals](/blog/completion-ranking-with-repository-signals)
• [Cursor, Copilot, Continue, Cody, Windsurf, Codeium: comparison framework](/blog/cursor-copilot-continue-cody-windsurf-codeium-comparison-framework)