Deploying a coding assistant to Kubernetes without leaking source

You can't just kubectl apply -f assistant.yaml and call it a day when source code is involved. A poorly configured AI coding assistant on Kubernetes will log repo contents to stdout, replicate secrets across ephemeral pods, and leave diffs sitting in terminating containers long after developers have moved on. Here's how to deploy one without turning your cluster into an inadvertent code-sharing platform.

The core threat model

AI coding assistants work by streaming context—file trees, diffs, git history, environment variables—to an LLM. On Kubernetes, that context flows through logs, volumes, ConfigMaps, and often an external model API. Each hop is a potential leak:

• Logs: The assistant's stdout/stderr may contain full file snippets when debugging a prompt or a failed lint run.
• Volumes: Shared PersistentVolumes can persist checked-out repos beyond pod lifetimes.
• Secrets: API keys for the LLM, git credentials, and registry tokens end up in environment variables that get inherited by crash-dumping sidecars.
• Network: If your assistant calls an external model endpoint, every prompt payload leaves your cluster boundary.

A secure deployment means deciding what can leave the cluster, what stays ephemeral, and what audit trail you need.

Ephemeral storage for transient repos

Start by ensuring cloned repositories never outlive the pod doing the work. Use emptyDir volumes with the Memory medium for small repos or SizeLimit constraints for larger ones:

volumes:

  - name: workspace

    emptyDir:

      medium: Memory

      sizeLimit: 512Mi

When the pod terminates, the cloned repo vanishes. This is critical for assistants that process pull requests or run batch code analysis—you don't want yesterday's feature branch still mounted when a new job starts.

If you need persistent storage for caching dependencies or build artifacts (node_modules, .venv), isolate it in a separate volume and never commit source to it. Goatfied's agent loop clones the target repo fresh each time, runs its plan/constrain/edit/validate cycle, then tears down, so you avoid drift between the cached state and the actual HEAD.

Logging without leaking diffs

Most Kubernetes logging stacks—Fluentd, Loki, CloudWatch—scrape all container stdout. If your assistant writes "Applied diff:\n```diff\n- const API_KEY = 'sk-...'" to stderr during a retry, that's now in your centralized logs forever.

Configure the assistant to redact or truncate code in logs. Goatfied emits structured JSON logs with fields like operation, file_path, and validation_result, but omits the actual diff content unless DEBUG_INCLUDE_DIFFS=true is set. In production, leave that off.

For operators who need to debug stuck runs, stream logs to a short-TTL S3 bucket or a tmpfs-backed volume that pods can write to but external scrapers ignore:

volumeMounts:

  - name: debug-logs

    mountPath: /var/log/assistant

volumes:

  - name: debug-logs

    emptyDir:

      sizeLimit: 100Mi

Keep the retention window under 24 hours and restrict bucket/volume access to the ops team. This gives you visibility during incidents without polluting long-term log indexes with proprietary code.

Secrets management and least privilege

AI assistants need credentials: a git token to clone private repos, an API key for the LLM, maybe a Jira token to post comments. Don't bake them into the container image or a shared ConfigMap. Use Kubernetes Secrets with strict RBAC:

apiVersion: v1

kind: Secret

metadata:

  name: assistant-secrets

  namespace: ai-workloads

type: Opaque

data:

  GIT_TOKEN: <base64>

  LLM_API_KEY: <base64>

---

apiVersion: v1

kind: ServiceAccount

metadata:

  name: assistant-sa

  namespace: ai-workloads

---

apiVersion: rbac.authorization.k8s.io/v1

kind: Role

metadata:

  namespace: ai-workloads

  name: assistant-role

rules:

  - apiGroups: [""]

    resources: ["secrets"]

    resourceNames: ["assistant-secrets"]

    verbs: ["get"]

The ServiceAccount should only access the specific secret it needs, nothing else. If your assistant also needs to create short-lived Jobs (e.g., to run tests in isolated pods), grant create on jobs but deny update and delete on unrelated resources.

For external LLM APIs, consider a sidecar proxy (Envoy, Nginx) that injects the API key from a Secret and strips it from responses. The main assistant container never sees the raw token, reducing the risk if the process is compromised or logs unexpectedly.

Network policies for model calls

If you're using an external LLM (OpenAI, Anthropic, a self-hosted endpoint outside the cluster), default Kubernetes networking allows any pod to reach any IP. Lock it down with a NetworkPolicy that permits egress only to the model API and your git server:

apiVersion: networking.k8s.io/v1

kind: NetworkPolicy

metadata:

  name: assistant-egress

  namespace: ai-workloads

spec:

  podSelector:

    matchLabels:

      app: goatfied-assistant

  policyTypes:

    - Egress

  egress:

    - to:

        - podSelector: {}

      ports:

        - protocol: TCP

          port: 53  # DNS

    - to:

        - ipBlock:

            cidr: 35.x.x.x/32  # example: your LLM endpoint

      ports:

        - protocol: TCP

          port: 443

    - to:

        - namespaceSelector:

            matchLabels:

              name: git-server

      ports:

        - protocol: TCP

          port: 443

This prevents accidental or malicious data exfiltration to arbitrary endpoints. If you run the LLM in-cluster (e.g., a quantized Llama on GPU nodes), you can lock egress down further to only intra-cluster communication.

Self-hosted LLMs and data residency

The most airtight deployment keeps the model inside your cluster. Goatfied supports self-hosted LLM backends; point it at a local vLLM or TGI server, and no code ever leaves Kubernetes. The tradeoff: you provision GPU nodes, manage model updates, and handle inference scaling yourself.

A minimal setup looks like a Deployment for the LLM server (with GPU tolerations and node affinity), a ClusterIP Service, and your assistant Deployment configured to call http://llm-service.ai-workloads.svc.cluster.local:8000. Since all traffic is in-cluster, you can enforce mTLS with service mesh policies (Istio, Linkerd) and never expose the LLM to the internet.

This approach also solves compliance requirements around data residency—if your cluster is in eu-central-1, the code and prompts stay in eu-central-1. Goatfied's compile/lint/test gates run locally too, so you're not sending half-baked diffs to a remote API for validation.

Audit trails for code modifications

When an AI assistant rewrites code, you need a record of what changed, who triggered it, and whether validation passed. Kubernetes audit logs capture API calls (pod creation, secret access) but not application-level edits.

Run a sidecar that watches the assistant's activity and writes events to an immutable log:

containers:

  - name: assistant

    image: goatfied/assistant:latest

    # ... main config

  - name: audit-sidecar

    image: your-org/audit-logger:v1

    volumeMounts:

      - name: audit-stream

        mountPath: /audit

volumes:

  - name: audit-stream

    emptyDir: {}

The sidecar tails a shared volume or connects to the assistant's internal event stream (if exposed via a Unix socket), then ships structured events—{timestamp, user, repo, files_changed, validation_status}—to a tamper-proof store (S3 with object lock, a blockchain-backed log, etc.). This gives you a cryptographically verifiable history separate from the code diffs themselves.

Goatfied emits events at each step of its agent loop (plan, constrain, edit, validate, retry), so you can trace exactly which constraints were active and why a particular edit was rejected or accepted.

Handling pod disruptions gracefully

Kubernetes will evict pods for node maintenance, resource pressure, or preemption. If an assistant is mid-edit when the pod gets a SIGTERM, you don't want it to push a half-applied diff. Configure a preStop hook that flushes state or aborts the operation:

lifecycle:

  preStop:

    exec:

      command: ["/bin/sh", "-c", "kill -TERM $(pidof assistant) && wait"]

Set terminationGracePeriodSeconds high enough (30–60s) for the assistant to roll back uncommitted changes and close git connections cleanly. Goatfied's small-diff strategy (one logical change per agent run) limits blast radius—if a pod dies mid-run, you lose at most one attempted edit, and the next pod retries from the validated HEAD.

For long-running assistants (e.g., watching a repo for new PRs), use a StatefulSet with a PersistentVolume for checkpoint data. When a pod restarts, it resumes from the last known good state instead of re-processing already-handled events.

Putting it together

A production-ready assistant deployment on Kubernetes layers ephemeral storage, secret isolation, network policies, and audit logging into a defense-in-depth posture. No single misconfiguration leaks your entire codebase, and operators get the visibility they need without exposing diffs to centralized log aggregators.

The cost is operational complexity: more YAML, stricter RBAC, custom sidecars. But if you're letting an AI touch production code, that complexity is table stakes. Goatfied's compile-first gates and reversible diff model make the runtime safer—validation happens before code leaves the pod, and failures don't cascade—but the infrastructure still has to enforce boundaries.

Related reading

• Self-hosted AI coding assistants: a complete Goatfied deployment guide
• Running AI coding assistants in air-gapped environments