Pull request descriptions engineers actually read

Every engineer has opened a PR to find the description field contains a single commit message, a Jira ticket number, or the dreaded "see title." Meanwhile, the PR itself touches twelve files, refactors two interfaces, and quietly fixes a race condition discovered during testing. The reviewer is left archaeology: reading every changed line to reconstruct intent, scope, and risk.

The problem isn't laziness. Writing good PR descriptions competes with finishing the feature, addressing CI failures, and context-switching to the next task. By the time you're ready to open the PR, you've already built the mental model—documenting it feels redundant. But that mental model evaporates for reviewers, and for your future self during incident triage.

Automation can help, but only if it produces descriptions that are actually read. Generic AI summaries trained to sound authoritative while saying little just add to the noise. What works is leveraging the artifacts you already create: commit history, test changes, and the constraints your tooling enforces.

What makes a description worth reading

A useful PR description answers three questions without requiring the reviewer to load the entire diff into their head:

What changed, in plain language. Not "refactored the auth layer" but "split session validation into a separate middleware so rate limiting can run before token parsing." Specific enough that a reviewer knows whether to skim or dig deep.

Why this approach, and what alternatives you rejected. If you considered using an external cache instead of in-memory state, mention it and why you didn't. This saves the reviewer from suggesting the same thing in comments.

Where to look for risk. Migration scripts, concurrency primitives, error handling changes—the places where subtle bugs hide. Pointing these out explicitly doesn't make you look uncertain; it makes you look thorough.

Automation that produces this kind of content requires access to more than commit messages. It needs to see test additions (what behavior is now validated?), lint/type changes (what contracts tightened?), and ideally what your build process enforced.

Mining signal from the agent loop

Goatfied's agent loop—plan, constrain, edit, validate, retry—creates a natural artifact trail that maps to useful description content. When the agent plans, it's articulating intent in semi-structured form. When it hits a type error or test failure and retries, you're seeing the decision tree: what didn't work, what adjustment fixed it.

This is richer than commit messages, which typically document the final state, not the path. A commit message says "add caching." The agent log shows "attempted Redis, hit connection pool limits in load tests, switched to LRU in-process cache with 10k entry cap."

You can extract this automatically:

// Simplified example of description generation

interface AgentTrace {

  plan: { intent: string; scope: string[] };

  attempts: { edit: Diff; validation: ValidationResult }[];

  finalDiff: Diff;

}



function generateDescription(trace: AgentTrace): string {

  const rejectedApproaches = trace.attempts

    .filter(a => !a.validation.passed)

    .map(a => `- Tried ${summarize(a.edit)}, failed: ${a.validation.error}`);

  

  return `

## Intent

${trace.plan.intent}



## Implementation

${summarize(trace.finalDiff)}



${rejectedApproaches.length > 0 ? `## Approaches ruled out\n${rejectedApproaches.join('\n')}` : ''}



## Validation

${trace.attempts.at(-1).validation.testsRun} tests passed

  `.trim();

}

The key is that validation failures aren't just noise—they're documentation of constraints you discovered. If the type checker rejected an any escape hatch, that tells the reviewer you tried to shortcut and the system caught you. That's a useful signal.

Diff semantics over LLM hallucination

The temptation with "AI-generated descriptions" is to feed the diff to a large language model and let it synthesize prose. This fails for two reasons:

First, LLMs excel at sounding confident about code they don't execute. They'll describe a function as "thread-safe" when it's not, or claim a change "improves performance" based on vibes. Reviewers learn to distrust these summaries quickly.

Second, diffs have structure that LLM tokenization discards. Adding a test file isn't just "more lines"—it's a signal about what behavior is now covered. Removing an unwrap() in Rust or a ! non-null assertion in TypeScript is a safety improvement you can detect statically.

Better approach: parse the diff semantically, emit structured facts, then template them into readable prose.

# Detect specific patterns

$ goatfied diff analyze feature-branch

Added error handling: 3 call sites now return Result<T>

Tightened types: removed 2 unsafe casts

New test coverage: auth/session.test.ts (+47 lines)

Migration required: schema version 003 -> 004

Now you have concrete, checkable claims. A human (or template) can turn these into "This PR adds error handling to three external API calls and removes unsafe type casts. Note the schema migration in migrations/004_add_session_index.sql—it requires a table rewrite for the sessions table."

No hallucination, no vague hand-waving about "improved robustness."

Inline context beats wall-of-text

Even a perfect description loses value if it's isolated in the PR body while the reviewer is scrolling through file diffs. The description should point to specific lines, but the risky lines should also be annotated inline.

This is where compile-time/test-time information can auto-inject comments:

// migrations/004_add_session_index.sql

+CREATE INDEX CONCURRENTLY idx_sessions_user_id ON sessions(user_id);

+-- AUTO: Estimated rewrite time for 10M rows: ~4min (non-blocking)

Or in code:

// src/auth/session.ts

-const user = JSON.parse(token);

+const user = sessionSchema.parse(token);

+// AUTO: Added runtime validation, will throw on malformed tokens

These annotations don't replace the PR description—they complement it. The description gives the high-level map; inline notes mark the terrain features worth examining.

Goatfied's agent can inject these automatically because it sees validation output. If a test suite shows a new assertion catching a malformed input, that's signal. If a linter flags a complexity increase, surface it.

The human override rule

Automation should never remove the human from the loop entirely. The best workflow:

1. Agent generates description from trace + diff semantics

2. Engineer reviews, adds context the agent can't infer (business rationale, customer impact, future plans)

3. Automation detects when the engineer edited description and doesn't overwrite on subsequent pushes

This matters because some context lives only in Slack threads, design docs, or your head. The agent might know "this changes the retry logic" but not "we're doing this because the payment provider's API is flaky on Tuesdays."

Preserve the edit field. Let engineers add nuance. Automation should be a starting point that raises the floor, not a ceiling that limits expression.

Measuring what works

You know your PR descriptions are useful when:

• Review time drops for unfamiliar code. If someone outside the immediate team can review without asking "what does this PR actually do?" you've succeeded.
• Post-merge questions decrease. Fewer "why did we do it this way?" questions three months later means the description captured intent.
• Incidents reference PRs directly. During an outage, if the team pulls up the PR and finds the risk section called out the exact failure mode, your automation is working.

Don't measure "percentage of PRs with descriptions"—that's a vanity metric. Empty compliance is worse than nothing because it trains people to ignore the field.

Related reading

• Inline diff UX that reduces review fatigue