Designing agent retry logic that doesn't burn your token budget

A failed agent attempt costs you more than tokens—it costs time, context window space, and the compounding error of retrying on top of broken code. Yet most agent frameworks treat retry logic as an afterthought: exponential backoff for rate limits, maybe a fixed attempt counter, and a prayer that the model will "just figure it out" on the next try.

The reality is harsher. Without structured retry logic, agents spiral. They re-attempt the same flawed edit, hallucinate fixes that introduce new failures, or blow through 50,000 tokens trying to recover from a single malformed import. The goal isn't to retry more—it's to retry smarter, with early exits, degraded fallbacks, and clear feedback loops that teach the agent what actually went wrong.

Why naive retry strategies waste tokens

The simplest retry loop looks like this: run the agent, check if it succeeded, and if not, run it again with a vague "please try again" message. This fails for three reasons.

First, you're re-sending the same input without new information. If the agent misunderstood the schema, the error message, or the constraint the first time, it will likely misunderstand it again. You've just doubled your token spend for the same outcome.

Second, context accumulates without pruning. Each retry appends the previous failed attempt to the conversation history. By attempt three, you're burning tokens on two failed edits, their error outputs, and the original prompt—none of which the model needs to see in full. The signal-to-noise ratio craters.

Third, there's no escape hatch for systemic issues. If the model is fundamentally incapable of the task (maybe the API changed and your prompt is stale, or the codebase uses a syntax the model doesn't recognize), retrying ten times won't help. You need a circuit breaker that says "this approach isn't working" and falls back to a human, a simpler task, or a no-op.

Structuring retries around validation gates

Goatfied's agent loop enforces a constraint-first flow: plan the change, validate it against known rules (types, lints, tests), edit the code, validate again, and only retry if validation fails with actionable feedback. This isn't optional—every edit passes through compile and lint gates before it's considered done.

Here's how that changes retry logic. Instead of a boolean "success/failure," you get a typed error with a specific failure mode:

• Syntax error at line 47: Retry with the exact error location and a prompt narrowed to "fix the syntax error on line 47; do not touch other files."
• Type mismatch in function signature: Retry with the type checker output, not the entire diff.
• Test failure in test_checkout.py: Retry with the test name, the assertion that failed, and the minimal context needed to understand the failure.

Each retry is scoped to the failure. If the agent introduced a syntax error, you don't ask it to re-plan the entire feature—you ask it to fix the syntax. This keeps the retry token cost proportional to the error, not the original task size.

Implementing token-aware retry budgets

A hard cap on retry attempts is better than nothing, but it's still crude. A better approach is a token budget per task, where each retry consumes from a shared pool and forces the agent to work within constraints.

For example, imagine a task starts with a 20,000-token budget. The initial plan and edit cost 8,000 tokens. The first validation fails (syntax error), and the retry costs 3,000 tokens. The second retry fixes it, costing another 3,000. Total: 14,000 tokens, well under budget. But if the agent is still failing after 18,000 tokens, the system should halt, log the failure, and either escalate or fall back.

You can track this with a simple counter:

budget_remaining = 20_000

for attempt in range(max_retries):

    response = agent.run(prompt, context)

    budget_remaining -= response.token_count

    

    if budget_remaining <= 0:

        return TaskResult.budget_exceeded(context)

    

    validation = validate(response.edit)

    if validation.passed:

        return TaskResult.success(response.edit)

    

    prompt = narrow_prompt(validation.error)

The key insight: token budgets enforce prioritization. The agent can't waste tokens on speculative refactors or overly verbose explanations. It has to fix the error and move on.

Narrowing context on each retry

When validation fails, the retry prompt should include:

1. The specific error message (compiler output, test failure, lint warning).

2. The minimal code context needed to understand the error (the function with the type error, not the entire file).

3. A constraint that prevents the agent from re-attempting unrelated changes ("fix only the import statement; do not modify the function body").

Here's what this looks like in practice. Original prompt:

> "Add a checkout function that validates the user's cart and processes payment."

After a type error in the retry:

> "Type error: checkout() expects Cart but received dict. The function signature is:

> ```python

> def checkout(cart: Cart, payment: PaymentMethod) -> Order:

> ```

> Fix the type mismatch in the function call on line 34. Do not change the function signature or add new imports."

This scoped retry uses a fraction of the tokens, and because it's hyper-specific, the model is far more likely to succeed on the first re-attempt.

When to degrade vs. when to abort

Not all failures are worth retrying. Some should trigger an immediate degraded fallback:

• Persistent test failures after two retries: Fall back to a simpler implementation (e.g., skip the edge case, log a warning instead of throwing an error).
• Compile errors in generated migration code: Fall back to a manual migration stub with comments indicating what the agent intended, so a human can finish it.
• Timeout on external API calls: Fall back to a cached response or a no-op, with a logged warning.

Others should abort outright:

• Three consecutive syntax errors in the same file: Something is fundamentally broken in the agent's understanding. Abort and escalate.
• Token budget exceeded before first validation pass: The task is too large or the prompt is too verbose. Abort and split the task.
• Identical failed edit produced twice in a row: The agent is looping. Abort and log the loop for debugging.

These rules prevent the agent from churning indefinitely. In Goatfied's self-hosted mode, you can customize these thresholds per project—some teams tolerate more retries for non-critical scripts, fewer for production API changes.

Example: retry logic for a refactor task

Let's walk through a realistic scenario. An agent is tasked with renaming a function across a Python codebase.

Attempt 1: The agent renames the function definition but misses two call sites. Validation runs tests; two tests fail with NameError: old_function_name is not defined. Token cost: 6,000.

Retry 1: The system narrows the prompt to "Fix the two remaining call sites in module_a.py and module_b.py. The correct function name is new_function_name." The agent updates both call sites. Validation passes. Token cost: 2,500.

Total: 8,500 tokens for a successful refactor. Compare this to a naive retry that re-sends the full prompt and diff: easily 15,000+ tokens, with no guarantee of success.

Logging retries for continuous improvement

Every retry is a learning opportunity. Goatfied logs:

• The original prompt and the retry prompt (to see if narrowing worked).
• The error that triggered the retry (to identify common failure modes).
• Token costs per attempt (to spot budget inefficiencies).
• Final outcome (success, degraded, aborted).

Over time, these logs reveal patterns. Maybe agents consistently fail on a specific linter rule—time to add an explicit constraint to the initial prompt. Maybe retry budgets are too generous for simple tasks—adjust the defaults. This feedback loop turns retry logic from a defensive mechanism into a tuning parameter.

Related reading

• The Goatfied agent loop: how we ship code that actually compiles first try
• Why AI agents fail on the second edit, and how validation loops fix it