Deterministic guards around non-deterministic tool calls

LLMs can call functions, query databases, and edit files—but their output is probabilistic. You can't guarantee a JSON blob will parse, that a tool call argument references a file that exists, or that ten concurrent edits won't conflict. Teams building AI agents discover this fast: the first demo works, the second run halts on a malformed parameter, and the third corrupts state because nothing checked whether user_profile.json was already being written to.

The gap between "the model chose to call a tool" and "the tool executed successfully" is where most agent failures live. Bridging it requires deterministic guards—checks that run before execution, after every response, and between steps. These aren't AI alignment problems; they're engineering problems with solutions from distributed systems and compilers.

The reliability gap in tool calling

When a model decides to use a tool, it emits a structured payload: function name, arguments, maybe metadata. Most frameworks stop there. They deserialize the JSON, look up the handler, and invoke it. If the model hallucinated a file path, used the wrong type for a numeric ID, or called two incompatible operations in sequence, you find out only after the crash—or worse, after partial state corruption.

Consider a file-editing agent. The model sees:

src/api/routes.ts

src/utils/logger.ts

tests/api.test.ts

It plans to refactor routes.ts and logger.ts in parallel. Without guards, both edits dispatch simultaneously. If routes.ts imports logger.ts and both change simultaneously, you can end up with syntax errors or import mismatches that neither edit alone would cause. The model didn't "intend" to break the code, but the tool execution layer had no mechanism to serialize conflicting writes.

The same pattern appears with API calls (rate limits, invalid credentials cached from a prior step), database writes (foreign key violations), or shell commands (clobbering files mid-read). Probabilistic tool selection meets deterministic infrastructure, and the latter wins every time.

Schema validation as the first gate

The cheapest guard is schema enforcement. Before invoking any tool, validate the call against a contract. OpenAI's function calling and Anthropic's tool use both return JSON, but the runtime doesn't enforce your schema—it just parses.

A minimal validation layer:

import { z } from "zod";



const EditFileSchema = z.object({

  path: z.string().regex(/^[a-zA-Z0-9_\-\/\.]+$/),

  start_line: z.number().int().positive(),

  end_line: z.number().int().positive(),

  new_content: z.string(),

});



function guardedEditFile(args: unknown) {

  const parsed = EditFileSchema.safeParse(args);

  if (!parsed.success) {

    return { error: parsed.error.format(), retryable: true };

  }

  // now invoke the real tool with parsed.data

}

This catches:

• Type mismatches (string IDs passed as numbers)
• Path injection attempts (../../etc/passwd)
• Nonsense ranges (end before start)

It doesn't catch:

• Whether the file exists
• Whether the range is valid for the current file length
• Whether another operation just modified the same file

Schema validation is a compile-time gate for runtime data. It's necessary but not sufficient.

Precondition checks: the file you think exists

After schema validation, check assumptions. If the tool expects src/config.ts to exist, verify it before handing off to the file writer. If the plan assumes a database table has certain columns, query INFORMATION_SCHEMA or your ORM metadata first.

Goatfied's edit loop runs preconditions on every file operation:

// before applying edits

const fileState = await fs.stat(path).catch(() => null);

if (!fileState) {

  return { error: `File not found: ${path}`, suggest_refresh: true };

}



const currentContent = await fs.readFile(path, 'utf-8');

if (currentContent.split('\n').length < end_line) {

  return { 

    error: `File has ${currentContent.split('\n').length} lines, edit targets line ${end_line}`,

    retryable: true 

  };

}

When a precondition fails, you have choices:

• Surface the error to the model with a clear message: "File missing, did you mean routes.tsx?"
• Auto-refresh context if the agent is working from stale state
• Reject and retry with an updated prompt

The key is failing before side effects. Once you've written bytes to disk or sent an API request, rollback gets expensive.

Concurrency: you can't edit the same file twice

File-editing agents often generate multi-step plans. Precondition checks protect against staleness, but they don't prevent race conditions. If two edits target overlapping lines, or one deletes a function another renames, sequential preconditions alone won't save you.

We use a simple locking mechanism: track in-flight operations by resource. Before dispatching a file edit, check the lock table:

const activeLocks = new Map<string, Set<string>>();



function acquireLock(resourceId: string, operationId: string): boolean {

  const existing = activeLocks.get(resourceId);

  if (existing && existing.size > 0) return false;

  

  activeLocks.set(resourceId, new Set([operationId]));

  return true;

}

If a lock fails, return an error: "File routes.ts is already being edited by operation abc-123. Retry when complete." The model's next turn can re-plan, wait, or pick a different file.

This is heavier-handed than Git-style merge, but for agent loops, simplicity beats sophistication. Small, serialized edits are easier to validate and retry than complex multi-file merges.

Post-execution validation: did it actually work?

Even with preconditions and locking, tools can fail. A file write succeeds but leaves the code unparseable. An API call returns 200 but the mutation didn't apply. The deterministic guard here is compile and test after every change.

Goatfied runs a validation step after each edit batch:

1. Syntax check: run the language's parser (TypeScript's tsc --noEmit, Python's ast.parse)

2. Lint: surface import errors, undefined variables, type mismatches

3. Unit tests if they're fast (<5s)

If any fail, the agent sees:

Validation failed after edit to src/routes.ts:

  - TypeScript error TS2304: Cannot find name 'UserRequest'

  - ESLint: 'logger' is defined but never used (line 42)

This becomes the input to the next turn. The model doesn't need to "know" it broke the code—it receives structured feedback and retries. The loop is:

plan → constrain (preconditions) → edit → validate → retry or proceed

The validation gate ensures you never accumulate broken state. Each iteration either leaves the code in a valid state or rolls back.

Self-healing retries vs. hard stops

Not all errors are retryable. A schema violation or missing API key should halt; a transient network timeout or a fixable syntax error should loop. Categorize failures:

• Fatal: missing credentials, invalid workspace config, unrecoverable corruption
• Retryable: syntax errors, file not found (maybe the model got the name wrong), linting issues
• Deferrable: slow tests, optional optimizations

When a retryable error occurs, append it to the agent's context with a clear action: "Fix the TypeScript error on line 14." Limit retry attempts (we cap at 3 per sub-task) to avoid infinite loops.

For fatal errors, stop and surface to the user. No amount of prompt engineering will fix a bad environment config. Deterministic guards clarify why something failed so humans can intervene cleanly.

Practical tradeoffs: speed vs. safety

Every guard adds latency. Schema validation is microseconds; precondition checks might query the filesystem or a database; post-execution compilation can take seconds. For prototypes, you can skip some. For production agent loops, you can't.

We've found the sweet spot:

• Always: schema validation, basic file existence checks
• Per-batch: compilation and linting after a set of related edits
• On-demand: full test suites only when the user requests or before deployment

The cost is worth it. An agent that takes 15% longer but produces valid, compiling code on the first try is vastly cheaper than one that runs instantly but requires two rounds of human fixes.

Related reading

• The Goatfied agent loop: how we ship code that actually compiles first try
• Why AI agents fail on the second edit, and how validation loops fix it