Prompt Patterns For Reliable Test Generation Production Playbook: practical systems guide

Most AI test generators fail in production because they treat prompts like search queries instead of contracts. A vague "write tests for this function" yields brittle assertions that hard-code implementation details, ignore edge cases your system actually hits, and break on the next refactor. The fix isn't better models—it's structured prompts that treat test generation as constrained code synthesis, not creative writing.

This playbook walks through four prompt patterns we've refined for reliable test generation at scale: **specification extraction**, **constraint anchoring**, **negative-case enumeration**, and **assertion decomposition**. Each pattern addresses a specific failure mode and includes concrete before/after examples you can adapt immediately.

Pattern 1: Specification extraction over behavior inference

**The problem:** Asking "generate tests for this function" forces the model to reverse-engineer intent from implementation. It will cover what the code *does*, not what it *should* do—so a bug in the original becomes an assertion in the test.

**The pattern:** Extract the specification first, then generate tests against that spec. This two-pass approach makes implicit requirements explicit and catches logic errors before they fossilize.

// First pass: extract spec

export async function processOrder(order: Order): Promise<Receipt> {

  // Prompt: "List the functional requirements this function must satisfy,

  // including edge cases, error states, and invariants. Format as a

  // bulleted spec."

  //

  // Expected output:

  // - Must validate order.items is non-empty before processing

  // - Must calculate total with tax applied to taxable items only

  // - Must reject orders exceeding customer credit limit

  // - Must generate idempotent receipt IDs

  // - Must handle DB write failures by rolling back inventory holds

}

Now the second prompt targets that spec: "Generate tests covering each requirement. For each test, cite the spec item it validates." The model writes against a specification, not the artifact of however you happened to implement it last Tuesday.

In Goatfied's agent loop, this fits naturally into the **plan** phase—the spec extraction becomes part of the changeset context, and the **constrain** phase ensures generated tests compile and link against the correct interfaces before any files change.

Pattern 2: Constraint anchoring with type witnesses

**The problem:** Generated tests often use simplified types or mock objects that don't match production data shapes. A test that passes with `{ id: 1, name: "test" }` may fail in production when that object carries 40 additional fields and nested references.

**The pattern:** Provide type witnesses—real or anonymized production examples—and require tests to construct inputs from those templates. The model learns the actual schema complexity.

// Provide this context in the prompt:

const productionOrderExample = {

  id: "ord_7x4k9",

  items: [{ sku: "WIDGET-A", qty: 2, taxable: true }],

  customer: { id: "cust_9m2", creditLimit: 5000 },

  shipping: { method: "express", address: { /* 15 more fields */ } },

  metadata: { source: "web", campaign: null }

};



// Prompt: "Generate tests using objects shaped like productionOrderExample.

// Vary field values, but preserve structure."

This prevents the common failure where a test mocks a "User" as `{ id: 1 }` when your production User carries authentication state, preferences, audit timestamps, and foreign keys the code path actually touches.

Type witnesses also surface incorrect assumptions: if your prompt includes an example and the model's generated test immediately strips half the fields, that's a signal the function signature or your test setup is too permissive.

Pattern 3: Negative-case enumeration from error paths

**The problem:** Models default to happy-path coverage. They'll test `add(2, 3) === 5` but skip `add(null, 3)` or `add(2, Infinity)` unless you explicitly prompt for error cases—and even then, the coverage is generic.

**The pattern:** Walk the implementation's error paths first, enumerate failure modes, then generate one test per mode. Mechanically: identify every `throw`, `return null`, or error-state branch, and require a test that exercises it.

// Prompt structure:

// 1. "Identify all error paths in this function (throw statements, early

//    returns with error payloads, validation failures). List them."

// 2. "For each error path, generate a test that triggers it. Use realistic

//    inputs that production might see."



export function withdraw(account: Account, amount: number) {

  if (amount <= 0) throw new Error("Invalid amount");

  if (amount > account.balance) throw new Error("Insufficient funds");

  if (account.frozen) throw new Error("Account frozen");

  // ... happy path

}



// Expected test shape:

it("throws on negative withdrawal amount", () => { /* ... */ });

it("throws on amount exceeding balance", () => { /* ... */ });

it("throws when account is frozen", () => { /* ... */ });

This pattern pairs well with Goatfied's **validate** phase: the generated tests run in a sandbox, and failures surface quickly without blocking unrelated edits. If a negative-case test is flaky or wrong, the agent can retry with additional constraints ("this path should succeed when X, fail when Y") before merging.

Pattern 4: Assertion decomposition for debuggability

**The problem:** Generated tests often produce single monolithic assertions: `expect(result).toEqual({ /* 50-line object */ })`. When this fails, you get a 200-line diff and no clue which field broke or why.

**The pattern:** Decompose assertions into granular checks, each testing one invariant. The prompt explicitly requests "separate assertion per property" and explains the debugging benefit.

// Instead of:

expect(processOrder(order)).toEqual({

  receiptId: "rec_abc123",

  total: 105.50,

  tax: 5.50,

  items: [{ sku: "WIDGET-A", price: 50 }]

});



// Prompt for:

const receipt = processOrder(order);

expect(receipt.receiptId).toMatch(/^rec_[a-z0-9]+$/);

expect(receipt.total).toBe(105.50);

expect(receipt.tax).toBe(5.50);

expect(receipt.items).toHaveLength(1);

expect(receipt.items[0].sku).toBe("WIDGET-A");

Decomposed assertions make CI logs actionable. Instead of "expected object to equal object," you see "expected tax to be 5.50, got 6.00"—root cause in one line.

In practice, this pattern also catches under-specified requirements: if the model struggles to decompose an assertion, the function likely returns an overly complex object that should be split or the test needs domain-specific matchers.

Iterative refinement in the agent loop

These patterns aren't one-shot prompts—they're checkpoints in Goatfied's plan → constrain → edit → validate cycle. After generating tests, the **constrain** phase runs linters and type checkers; failures feed back as additional prompt context ("previous attempt failed type-check because X, adjust inputs accordingly"). The **validate** phase runs the tests; flakes or false positives trigger a retry with tighter specifications.

This loop converts brittle test generation into a ratchet: each iteration preserves the constraints that worked and refines those that didn't. The result is a test suite that reflects actual requirements, not a model's best guess at what "good coverage" looks like.

Small reversible diffs help here—if a generated test breaks an unrelated module, the agent can isolate the change and revert that test without discarding the entire batch.

Related reading

• [Why we open-sourced Goatfied (and what we kept proprietary)](/blog/why-we-open-sourced-goatfied-and-what-we-kept-proprietary)
• [Open-Source playbook #3: practical Goatfied tactics for shipping PRs](/blog/goatfied-open-source-playbook-3)