Secret Detection Before Agent Execution Implementation Guide: practical systems guide

Every AI coding agent eventually reaches for a secret. An API key tucked in a config file, a database password in an environment variable, a Slack webhook URL someone committed six months ago. The moment you let an agent write or modify code, you've opened a door to credential leakage—not through malice, but through the same autopilot mistakes human developers make, amplified by an agent's willingness to pattern-match anything that looks like working code.

The good news: secret detection is a solved problem at the commit boundary. Tools like `git-secrets`, `detect-secrets`, and `trufflehog` have spent years cataloging patterns and heuristics. The challenge is moving that gate *before* the agent loop ever touches production, and doing it in a way that doesn't add 30 seconds of latency to every edit.

Why pre-execution detection matters more than post-commit scanning

Post-commit secret scanning catches problems after they've entered version control. Even if you block the push, the credential has already been written to disk, potentially logged, and definitely exists in your agent's context window. If your agent is self-hosted, this might be acceptable. If you're running managed infrastructure or sharing logs for debugging, that window of exposure matters.

Pre-execution detection means the agent never writes the secret to a file in the first place. The edit is rejected before `git add`, before the compiler sees it, before it lands in any audit log. This is especially valuable when agents are iterating quickly—generating three variations of a config file in rapid succession—because a single missed secret could propagate across multiple retries before a human reviews the PR.

The tradeoff: you need detection that runs in milliseconds, not seconds, because it sits in the critical path between plan and edit. You can't afford to shell out to a full-repository scanner on every proposed diff.

Pattern libraries vs. entropy analysis: picking your detection strategy

Most secret detection tools use two complementary approaches. Pattern matching catches known formats: AWS keys starting with `AKIA`, GitHub tokens with `ghp_` prefixes, generic `password=` assignments. Entropy analysis flags high-randomness strings that *look* like secrets even without a known pattern—a 32-character alphanumeric blob in a JSON file probably isn't lorem ipsum.

For pre-execution detection, start with patterns. They're fast (regex against a few hundred lines of diff), deterministic, and produce few false positives if your patterns are tight. Keep a curated set of regexes for your stack: whatever cloud provider you use, your CI token formats, any third-party API patterns you know you integrate with.

Entropy-based detection is powerful but noisy. A base64-encoded test fixture or a randomly generated UUID will trigger it. In the agent loop, where the agent can't always distinguish between "this is test data" and "this is a real credential," you'll spend more time tuning thresholds or maintaining allowlists than you gain in coverage. Use entropy scanning as a secondary gate—run it on the full diff before opening the PR, not on every single edit attempt.

Integrating detection into the constraint phase

In Goatfied's agent loop, the **constrain** step happens after planning but before the agent writes any code. This is where you inject rules: "Don't modify files in `/vendor`," "Keep functions under 50 lines," "No direct database access in controllers." Secret detection fits naturally here as a constraint: "No content matching these secret patterns may appear in any proposed edit."

Implement this as a pre-edit validator. Before the agent writes a diff, run the proposed content through your pattern library. If a match is found, reject the edit immediately and return a message explaining *why*: "Proposed change to `config.yml` contains a pattern matching an AWS access key (line 14). Use environment variable injection instead." The agent can then retry with a different approach—often by factoring the credential into an env var or a secrets manager reference.

def validate_no_secrets(proposed_diff: str) -> ValidationResult:

    patterns = load_secret_patterns()  # AWS keys, GH tokens, etc.

    for line_num, line in enumerate(proposed_diff.splitlines(), 1):

        for pattern in patterns:

            if pattern.matches(line):

                return ValidationResult(

                    valid=False,

                    message=f"Line {line_num}: matches {pattern.name}. "

                            "Use environment variables or a secrets manager."

                )

    return ValidationResult(valid=True)

This runs in single-digit milliseconds for typical diffs. No external process, no filesystem I/O. Just regex against the string buffer.

Handling false positives and placeholder values

Agents love to generate example code with placeholder secrets. `api_key = "your_api_key_here"` or `password = "replace_me"`. These trigger pattern matchers but aren't real leaks. You have two options: maintain an allowlist of known placeholders, or teach the agent to use obviously fake formats.

The allowlist approach is brittle—you'll chase new placeholder variations forever. Better: in your constraint message, include an example of an acceptable pattern. "Use `api_key = os.environ.get('API_KEY')` instead of hardcoding." Most agents will pick up on this and apply it to future edits in the same session.

For test files, consider relaxing detection entirely. If the agent is writing to `tests/fixtures/` or `spec/`, and your test secrets are rotated dummy values anyway, the risk is lower. But be explicit about this in your validation logic—don't silently skip all test directories, document the exception and audit it periodically.

Layering in commit-time validation

Pre-execution detection isn't a replacement for commit-time scanning; it's a first line of defense. Before the agent opens a PR, run a full-repository scan with something like `detect-secrets` or `gitleaks`. This catches anything that slipped through: secrets in files the agent didn't modify but pulled from a template, or high-entropy strings that only look suspicious in aggregate.

In Goatfied, this happens in the **validate** phase after all edits are complete. The agent has generated a diff, the code compiles, tests pass, and now you scan the entire changeset. If a secret is found here, the agent retries from the plan step with additional context: "Scan detected credential in `config/database.yml`. Refactor to use environment variables before proceeding."

This two-layer approach—fast patterns in the edit loop, comprehensive scanning before commit—gives you speed without sacrificing coverage.

Related reading

• [The Goatfied agent loop: how we ship code that actually compiles first try](/blog/goatfied-agent-loop-compiles-first-try)
• [Agent-Loop playbook #1: practical Goatfied tactics for shipping PRs](/blog/goatfied-agent-loop-playbook-1)