Goatfied blog

Benchmarks, deployment notes, and hard-won patterns

Developer-to-developer writing. No fluff. Just what worked, what failed, and what shipped.

benchmarks

Measuring ghost-text latency honestly

Learn how to accurately measure AI code completion latency across all four components that affect the delay between typing and seeing suggestions.

2026-08-078 min read

benchmarks

A reproducible methodology for benchmarking multi-file refactors

A reproducible methodology for benchmarking AI refactoring tools using auditable pass criteria, version control, and automated validation across real repositories.

2026-08-068 min read

benchmarks

Why SWE-bench scores don't predict daily developer productivity

SWE-bench scores measure isolated patch generation on open-source issues, not the multi-file refactoring and codebase navigation that dominates real development work.

2026-08-058 min read

benchmarks

Latency versus accuracy in inline completion engines

Inline code completion engines must balance response time under 100ms with suggestion accuracy, as latency below 80ms matters more than high acceptance rates.

2026-08-048 min read

benchmarks

Measuring AI code quality beyond “does it compile”

Learn how to evaluate AI-generated code for production readiness using metrics beyond compilation, including performance impact, architectural patterns, and team conventions.

2026-08-038 min read

benchmarks

We pointed six AI coding tools at a 400k-line monorepo. Here's what broke.

We tested six AI coding assistants on a 400,000-line Python monorepo with real feature tasks to document how they handle cross-service changes at scale.

2026-08-028 min read

Benchmarks, deployment notes, and hard-won patterns

Filter by category

Filter by tag

Measuring ghost-text latency honestly

A reproducible methodology for benchmarking multi-file refactors

Why SWE-bench scores don't predict daily developer productivity

Latency versus accuracy in inline completion engines

Measuring AI code quality beyond “does it compile”

We pointed six AI coding tools at a 400k-line monorepo. Here's what broke.