Test generation that targets real coverage gaps

Most AI test generators churn out dozens of tests that exercise code you already covered, leaving the edge cases and integration boundaries—the places bugs actually live—completely untested. A team inherits a 10,000-line service with 40% line coverage, runs an AI tool overnight, wakes up to 85% coverage and 300 new tests, then ships a feature that crashes in production because none of those tests checked what happens when the database connection pool is exhausted mid-transaction.

Real coverage isn't counting lines visited. It's identifying the branches, states, and interaction patterns that aren't exercised, then generating tests that deliberately target those gaps. This means parsing existing tests to understand what's already validated, analyzing the codebase to find uncovered paths, and synthesizing new tests that fill specific holes rather than restating what you already know works.

Why line coverage misleads

Line coverage tools report a single number—78%, 92%—that masks enormous variation in risk. A file might be 100% covered but only test happy paths: valid inputs, successful responses, connections that never time out. The error-handling branches that wrap every external call? Untouched. The retry logic when a third-party API returns 429? Never executed. The fallback when the cache is cold and the database is slow? Uncovered.

Branch coverage improves on this by tracking whether both sides of every if were taken, but it still doesn't tell you whether you tested the right combinations. A payment flow might branch on user.verified and cart.total > 100 and payment_method.type == 'card'. You can hit every branch individually—one test where user.verified is true, another where cart.total > 100, a third where payment_method.type == 'card'—and still never test the scenario that matters: an unverified user trying to make a large purchase with a card. That's the combination that triggers a security check, and it's completely untested despite 100% branch coverage.

Mutation testing gets closer: it injects faults into your code (flip a > to >=, change && to ||) and checks whether your tests catch them. If they don't, you have a coverage gap. But mutation testing is slow, generates hundreds of variants, and doesn't directly suggest what test to write—it just tells you something is missing.

Mapping the coverage gap from existing tests

Before generating new tests, parse what's already there. Extract the actual inputs, assertions, and code paths each test exercises. A Django test suite might have:

def test_create_order_success(self):

    user = User.objects.create(email='test@example.com', verified=True)

    product = Product.objects.create(price=50)

    order = Order.create_order(user, [product])

    assert order.status == 'pending'

This tells you: the existing tests cover the happy path for a verified user buying a single product under $100. They check that order.status ends up as pending but don't assert anything about payment processing, inventory reduction, or notification triggers.

Now scan the Order.create_order implementation. You find branches for:

• Unverified users → raises UnverifiedUserError
• Empty cart → raises EmptyCartError
• Out-of-stock products → partial order creation with backorder flag
• High-value orders (>$500) → triggers fraud check
• Payment gateway timeout → retries twice, then marks order as payment_pending

None of those branches appear in the existing test corpus. That's your coverage gap, specified not as "line 47 untested" but as "we never validated error handling when the user is unverified, never checked inventory reduction, never simulated a payment timeout."

Generating tests that target specific gaps

An LLM with access to the implementation, the existing test patterns, and the coverage analysis can propose tests like:

def test_create_order_unverified_user_rejected(self):

    """Verify that unverified users cannot place orders."""

    user = User.objects.create(email='test@example.com', verified=False)

    product = Product.objects.create(price=50)

    with pytest.raises(UnverifiedUserError):

        Order.create_order(user, [product])

This isn't randomly synthesized—it's generated because the LLM identified that the if not user.verified branch was never taken in the existing suite. Similarly, for the payment timeout path:

@patch('payment_gateway.process_payment')

def test_create_order_payment_timeout_retries(self, mock_process):

    """Ensure payment timeouts trigger retry logic."""

    mock_process.side_effect = [Timeout(), Timeout(), {'status': 'completed'}]

    user = User.objects.create(email='test@example.com', verified=True)

    product = Product.objects.create(price=50)

    order = Order.create_order(user, [product])

    assert mock_process.call_count == 3

    assert order.status == 'confirmed'

The test explicitly targets the retry loop that wasn't exercised before. It's not guessing—it's responding to the fact that the code has retry logic and no existing test validates it.

Constraints that keep generated tests realistic

Unconstrained LLM generation produces tests that look plausible but fail in practice: they call methods that don't exist, mock objects incorrectly, or assert on internal state that's not exposed. Constraints narrow the solution space to tests that actually compile and integrate with your test harness.

Start with the schema of your test framework. If you're using pytest, the LLM needs to know:

• Fixtures are defined with @pytest.fixture and injected by name
• Database setup often happens in conftest.py or a db_session fixture
• Mocking uses unittest.mock.patch or pytest-mock
• Assertions are plain assert statements, not assertEqual

Then layer in project-specific patterns. If your team always uses a create_test_user() helper, the generated tests should call that instead of invoking User.objects.create() directly. If you have a custom @with_temp_redis decorator for tests that need a cache, the LLM should apply it when generating tests for cache-dependent code.

The Goatfield agent loop enforces this by running lint and test validation after each generation step. If the generated test doesn't compile or uses an undefined fixture, the agent sees the error, adjusts the test, and tries again. This produces tests that match your project's conventions without requiring you to manually enumerate every pattern.

Integration tests for cross-service gaps

Unit tests cover individual functions, but coverage gaps often appear at boundaries: the interaction between a web handler and a background task, a cache miss that triggers a database query, a webhook that updates state in two services. These require integration tests that stand up multiple components and validate their interaction.

For a REST API that queues background jobs, the coverage gap might be "we never tested what happens if the job queue is full." The generated test could:

def test_api_job_queue_full_returns_503(client, job_queue):

    job_queue.set_capacity(0)  # Simulate full queue

    response = client.post('/api/orders', json={'user_id': 1, 'product_id': 2})

    assert response.status_code == 503

    assert 'queue unavailable' in response.json()['error']

This requires understanding that the /api/orders endpoint enqueues a job, the job queue can be full, and the expected behavior is returning 503 rather than silently failing or blocking forever. The test exercises a failure mode that unit tests don't cover because they mock the queue away.

Similarly, for a service that caches API responses, the gap might be "we never tested cache invalidation when the upstream data changes." The generated test sets up both the cache and a mock upstream, makes a request that populates the cache, triggers an update event, and verifies that the next request fetches fresh data instead of serving stale cache.

Iterating on coverage targets

Coverage improvement isn't one-shot. You run generation, validate the new tests, merge them, and the coverage map updates. Now you have new gaps: the tests you just added revealed branches in helper functions that weren't exercised before, or they covered some but not all error conditions in a subsystem.

Set a target—"bring order_service.py to 90% branch coverage"—and iterate. After each round, the agent identifies remaining gaps: "The apply_discount function has an untested path when discount.expired_at < now(), and the bulk order flow doesn't validate inventory for items added after the initial check." Generate tests for those, validate, merge, repeat.

This works because the constraints and validation ensure each round produces tests that integrate cleanly. You're not accumulating a backlog of half-working test stubs; every generated test compiles, runs, and either passes or fails with a clear error that leads to a refinement.

Related reading

• Pull request descriptions engineers actually read
• Risk-scoring pull requests for smarter review triage