Prompt Patterns For Reliable Test Generation Implementation Guide: practical systems guide

# Prompt Patterns For Reliable Test Generation Implementation Guide: practical systems guide

Generating useful tests with LLMs is harder than it looks. The model will happily produce dozens of test cases that compile, run green, and assert nothing meaningful. After shipping our first agent-written test suites into production, we learned that reliability comes from constraining the generation process itself—not from hoping the model "understands" what makes a good test.

This guide walks through the prompt patterns and validation gates we've built into Goatfied's test generation flow, focusing on the mechanics that prevent low-value output and force the agent to produce tests that actually catch regressions.

Start with the diff, not the entire codebase

The biggest mistake in test generation is asking the LLM to "write tests for this function." You get back generic happy-path cases that would've passed before your change and will pass after it.

Instead, anchor the prompt on the specific diff. Show the model what changed—new branches, refactored error handling, added edge cases—and ask it to generate tests that exercise those changes. This narrows scope and gives the agent a concrete target.

You are generating tests for this change:



{{ git diff }}



Focus on:

- New conditional branches introduced in lines 47-52

- The error case added for null paymentMethod

- Changed return type from string to Result<string, PaymentError>



Generate test cases that would fail on the old implementation but pass on the new one.

This pattern produces tests that are directly tied to your work. If the diff adds a null check, the agent writes a test with a null input. If you refactor an exception into a Result type, it generates assertions for both Ok and Err variants.

Require explicit assertions before code

The second pattern forces the agent to describe what it's testing before writing any code. We break generation into two steps:

**Step 1: Assertion plan**

For each test case, write:

1. Test name (should_X_when_Y format)

2. Input values

3. Expected output or exception

4. Why this case matters (one sentence)



Do not write code yet.

**Step 2: Implementation**

Now implement the test cases above. Each test must include:

- Setup with the exact inputs you specified

- One assertion matching your expected output

- No try-catch blocks unless you're testing exceptions

Separating the plan from the code prevents the agent from generating a test body first and backfilling an assertion that matches whatever the code happens to return. It's easy to spot vague plans ("test edge cases") and reject them before any code is written.

Enforce coverage boundaries with compile-time gates

Generic prompts produce generic coverage. "Write comprehensive tests" results in ten variations of the happy path. Instead, use the language's type system and your existing build tools to define coverage boundaries.

For Rust, we pass the agent a list of uncovered match arms from `cargo-llvm-cov`:

These branches are not covered by existing tests:



src/payment.rs:47 - PaymentMethod::Crypto(address) match arm

src/payment.rs:89 - Err(InvalidCurrency) branch



Generate one test per uncovered branch. Each test must execute that branch and assert on its output.

For TypeScript, we use the project's existing Jest coverage config and feed the agent a filtered list of uncovered lines:

// .goatfied/coverage-requirements.json

{

  "enforced_functions": ["processPayment", "validateAddress"],

  "min_branch_coverage": 85

}

Then in the prompt:

Your tests must increase branch coverage for these functions to 85%.

Current coverage: processPayment (62%), validateAddress (71%)

The key is to make coverage requirements concrete and machine-checkable. The agent loop's validation step runs `cargo test` or `npm run test:coverage` and rejects any generation that doesn't hit the target.

Use negative examples to prevent anti-patterns

LLMs are very good at copying patterns they've seen before. If your existing test suite has bad habits—mocking everything, testing implementation details, asserting on log output—the generated tests will inherit them.

We now include explicit negative examples in the prompt:

DO NOT generate tests that:

❌ Mock the function under test itself

❌ Assert on private methods or internal state

❌ Check log output instead of return values

❌ Use sleeps or arbitrary timeouts

❌ Test multiple unrelated behaviors in one case



Examples of bad tests to avoid:

{{ snippet of anti-pattern from real codebase }}

This pattern works better than positive-only instructions. Showing the model what not to do reduces the chance it drifts into familiar-but-wrong territory.

Validate with existing lint and test tooling

The plan → constrain → edit → validate loop is critical here. After the agent writes tests, run them through the same gates as human-written code:

1. **Lint**: Does the new test file pass your project's linter? Catches formatting, unused imports, overly complex test bodies.

2. **Compile**: Does it type-check? Catches mismatched types, missing imports, incorrect function signatures.

3. **Test**: Do the new tests actually run and pass? Catches syntax errors, broken assertions, uninitialized state.

4. **Coverage**: Do they increase coverage of the target diff? This is the final gate—if coverage doesn't move, reject and regenerate.

Each validation failure becomes part of the next prompt:

Your previous attempt failed at the test stage:



Error: Expected Result<String, PaymentError> but got String on line 42



Fix the assertion to match the actual return type.

Goatfied's agent loop retries with these error messages in context, typically resolving simple type mismatches within 1-2 iterations.

Small diffs, narrow scope, fast feedback

The final pattern is architectural: generate tests in small, focused batches tied to individual commits or logical changes. Don't try to backfill an entire module's test coverage in one agent run.

Break it down:

• One PR adds null handling → agent generates null-input tests
• Next PR refactors error types → agent generates error-case tests
• Next PR optimizes a hot path → agent generates performance regression tests

Each batch is small enough to review in a few minutes and cheap enough to regenerate if the first attempt misses the mark. This also keeps the validation loop fast—running 5 new tests takes seconds, running 50 takes long enough that you'll skip it.

Commit generated tests like any other code

Once a test batch passes all validation gates, commit it. Treat generated tests as first-class code: they go through PR review, get refined by humans if needed, and live in version control alongside everything else.

The agent's job is to do the tedious work of translating "this function now handles null" into a handful of passing test cases. Your job is to verify that those tests would actually catch a regression if someone reverts the change.

Related reading

• [Why we open-sourced Goatfied (and what we kept proprietary)](/blog/why-we-open-sourced-goatfied-and-what-we-kept-proprietary)
• [Open-Source playbook #3: practical Goatfied tactics for shipping PRs](/blog/goatfied-open-source-playbook-3)