Telemetry in open-source developer tools, done ethically

Most developer tools that phone home do it wrong. They bundle analytics SDKs that track everything, bury consent in 47-page privacy policies, and treat "telemetry" as a euphemism for "we collect what we want." When the tool is open-source, this gets worse: users expect transparency but find obfuscated endpoints and vague data-retention promises. The ethical gap isn't just bad PR—it actively undermines trust in the communities these projects depend on.

We hit this head-on when open-sourcing parts of Goatfied. Our cloud platform needs usage data to improve agent reliability and catch failure modes early. Our self-hosted runtime gives teams full control but no default metrics. Bridging that divide without becoming surveillance-ware required rethinking telemetry from the ground up: explicit opt-in, readable payloads, and architecture that makes invasive tracking structurally difficult.

Start with explicit opt-in, not dark patterns

The standard playbook is opt-out: enable telemetry by default, mention it in release notes, and let users hunt for the off switch. This fails the ethics test because inertia is not consent. If most users never notice the checkbox, you're exploiting laziness, not earning trust.

Explicit opt-in means the tool asks once, clearly, and remembers the answer. For Goatfied's open-source agent runtime, the first-run wizard presents:

Telemetry helps us improve reliability and catch edge cases.

We collect: agent loop outcomes, constraint violations, retry counts.

We never collect: code diffs, file paths, or user identifiers.



Send anonymous telemetry? (Y/n)

If the user skips or says no, we drop the flag in ~/.config/goatfied/telemetry.toml:

[telemetry]

enabled = false

version = "2024-12-01"  # schema version for compatibility

The schema version matters. When we change what we collect, we bump it and re-prompt on next upgrade. No silent expansion of scope.

Make payloads human-readable and introspectable

Binary analytics blobs are a red flag. If a developer can't inspect what you're sending—ideally with tcpdump or a proxy—you've failed the transparency test. JSON over HTTPS is table stakes, but even that isn't enough if the schema is opaque.

Our telemetry endpoint accepts line-delimited JSON events. A sample agent-loop completion looks like:

{

  "event": "agent_loop_complete",

  "timestamp": "2024-12-15T10:23:45Z",

  "runtime_version": "0.4.2",

  "loop_id": "uuid-here",

  "outcome": "success",

  "iterations": 3,

  "constraints_violated": ["clippy::pedantic"],

  "validation_gates": ["cargo:check", "cargo:test"],

  "retry_reason": null

}

Notice what's not here: repository names, file paths, code snippets, or user identifiers. The loop_id is session-scoped and regenerated on restart. If you grep our source for where this gets built, you'll find one telemetry module with explicit field allowlists—no "capture everything and filter later" logic that could leak sensitive data in a future refactor.

We also log outbound telemetry locally before sending, so users can audit:

$ tail ~/.cache/goatfied/telemetry.log

{"event":"agent_loop_complete","outcome":"constraint_violation",...}

If you're uncomfortable with a field we collect, file an issue. If it's not essential for reliability work, we'll remove it.

Separate telemetry from crash reporting

Crash reports are a special case. When the agent loop panics, you want stack traces and environment details to debug. But bundling that with routine telemetry conflates two trust models: operational metrics (low-sensitivity, high-volume) versus diagnostic dumps (high-sensitivity, rare).

Goatfied treats them separately. Telemetry is always opt-in and anonymized. Crash reporting requires a second consent dialog, warns that it may include code context, and uploads to a distinct endpoint with shorter retention (7 days versus 90 for telemetry). The prompt is explicit:

The agent runtime crashed. Send diagnostic report?

May include: stack trace, recent agent logs, workspace language.

Will NOT include: code diffs or credentials.



Upload crash report? (y/N)

Default is no. The report gets written to /tmp/goatfied-crash-<uuid>.json regardless, so users can inspect before deciding to send. For air-gapped or compliance-sensitive environments, this separation is non-negotiable.

Use aggregation and differential privacy where possible

Even with clean schemas, high-cardinality fields can become fingerprints. If you log every unique constraint name or test pattern, you've created a side channel for identifying projects. Aggregation helps: instead of sending raw counts, bucket them (iterations: 1–3, iterations: 4–10, iterations: 11+).

For fields that must stay high-resolution—like specific clippy lint violations that help us prioritize fixes—we apply differential privacy noise. Before sending an event batch, we randomly drop 5% of entries and inject 1% synthetic decoys. This prevents exact reconstruction of a single user's activity even if our database leaks. It's a small reliability cost (noisier metrics) for a large ethical gain.

This isn't overkill. Open-source projects have hostile forks, supply-chain attackers, and nation-state adversaries. If your telemetry backend gets compromised, granular event streams become exploit reconnaissance. Aggregation and noise shrink the blast radius.

Document retention and let users request deletion

"We keep your data as long as necessary" is lawyer-speak for "forever unless you sue." Ethical telemetry has defined retention and respects deletion requests, even when technically challenging.

For Goatfied, telemetry events live 90 days in hot storage, then get aggregated into summary statistics and purged. Crash reports are 7 days only. We publish these numbers in TELEMETRY.md in the repo root, next to the exact SQL schema and retention cron jobs. If we need to extend retention for a specific investigation, we announce it in release notes and re-prompt active users.

Deletion requests are tricky with anonymized data—there's no user ID to key off. Our compromise: we accept requests by IP + time window. If you email privacy@goatfied.com with "delete telemetry from 203.0.113.42 between Dec 10–15," we drop matching records. It's not perfect (shared IPs, VPNs) but it's honest effort within architectural constraints.

Provide local-only alternatives for sensitive environments

Some teams can't send telemetry, period. Air-gapped networks, regulated industries, or simple policy. For them, Goatfied's self-hosted runtime includes a local metrics exporter:

[telemetry]

enabled = true

destination = "local"

export_path = "/var/log/goatfied-metrics"

format = "prometheus"  # or "json", "csv"

Events get written to disk in the same schema as our cloud backend, but never leave the host. Teams can ingest them into their own observability stack or ignore them entirely. This design choice—making local telemetry first-class, not a fallback—signals that we respect their operational boundaries.

It also future-proofs our own architecture. If we ever need to pivot our cloud telemetry provider, the export format stays stable. Users don't eat breaking changes because we switched vendors.

Governance: who decides what gets collected?

The hardest ethical question isn't "how do we collect data" but "who decides what's acceptable?" For proprietary tools, the answer is simple and unsatisfying: the company. For open-source projects, community governance matters.

When we open-sourced our agent runtime, we created a telemetry-review label for GitHub issues. Proposals to add new fields require:

1. Concrete reliability problem it solves (no "might be useful someday")

2. Privacy impact assessment (can it fingerprint users?)

3. Two maintainer approvals and a 7-day comment period

We've rejected proposals for collecting editor theme (fingerprinting risk) and repository size (not actionable for reliability). We've accepted retry-after-validation outcomes (helps tune our constraint engine) and compilation error categories (guides LSP improvements).

This slows feature velocity but keeps scope creep in check. Trust compounds over time; blow it once with overreach and you're done.

Related reading

• Why we open-sourced Goatfied (and what we kept proprietary)
• What we learned open-sourcing our agent runtime