agent-platform

A 5-agent pipeline that detects production crashes, grounds the diagnosis against the actual repo, generates a patch, validates it in a Docker sandbox, and opens a PR — with a human approval gate before any merge, escalating urgency for HIGH/CRITICAL severity.

Request a walkthrough → Source on GitHub →

Pipeline

Every production error flows through this graph. Sandbox failure regenerates the fix up to 3× before any GitHub noise. HIGH/CRITICAL incidents stop at the approval gate.

flowchart TB A([CloudWatch Logs poll · 5 min]) --> D{Dedup gate} D -- duplicate --> X([drop]) D -- new --> E[TriageAgent · Haiku] E -- noise --> X E -- duplicate --> X E -- real, P0–P3 --> F[DiagnosisAgent · Sonnet] F --> G[FixGenerationAgent · Sonnet] G --> SC[Self-critique · Haiku] SC -- LIKELY WRONG --> G SC -- pass --> H{Sandbox npm test} H -- fail · retry 3× --> G H -- pass --> J[Open GitHub PR] J --> I[CodeReviewAgent · GPT-4.1] I --> K{Risk} K -- LOW/MED --> L[Human approval] K -- HIGH/CRIT --> L L --> M[Merge] classDef event fill:#0b0d10,stroke:#2f343b,color:#e6e8eb classDef agent fill:#13161a,stroke:#6ee7b7,color:#e6e8eb classDef gate fill:#0b0d10,stroke:#23272d,color:#9aa1a9 class A,X event class E,F,G,SC,I agent class D,H,K gate

End-to-end pipeline — from detected event to merged fix PR.

Deployment topology

Cloudflare-fronted ALB sits in front of ECS Fargate. Postgres + pgvector is the source of truth for incidents and the RAG index. CI/CD is OIDC — no secrets in the repo.

flowchart LR U([User · Interviewer]) --> CF[Cloudflare edge] CF --> ALB[AWS ALB :443] ALB --> ECS[ECS Fargate task FastAPI + Uvicorn :8000] ECS --> PG[(Postgres + pgvector)] ECS --> S3[(S3 artefacts)] GH[GitHub Actions OIDC] -. on push to main .-> ECR[ECR] ECR -. ECS pull .-> ECS CWLOGS[(CloudWatch Logs target app)] -. poll · 5 min .-> ECS classDef ext fill:#0b0d10,stroke:#23272d,color:#9aa1a9 classDef compute fill:#13161a,stroke:#6ee7b7,color:#e6e8eb classDef store fill:#13161a,stroke:#2f343b,color:#e6e8eb class U,CF,GH ext class ALB,ECS compute class PG,S3,ECR,CWLOGS store

AWS ECS Fargate · Cloudflare · GitHub Actions OIDC.

The agents

IncidentResponseAgent

Orchestrator + tool chain

Orchestrates the full pipeline. Receives error events, runs dedup checks, coordinates Triage → Diagnosis → Fix in sequence, and manages the approval gate.

TriageAgent

ReAct + typed wrapper

Classifies events as real / noise / duplicate. P0–P3 severity. Haiku for cost.

DiagnosisAgent

ReAct + grounding guards

Roots every claim against the live repo via GitHub Code Search. Retrieves relevant code context via hybrid search (α=0.7 vector + 0.3 lexical, min_score=0.45 quality floor).

FixGenerationAgent

Direct LLM + critique + sandbox

Writes the patch (Sonnet). Haiku self-critique rejects bad fixes before they hit the sandbox. Retries 3× on test failure. State pruning removes stale read_file results every 3 iterations (~18K tokens per run).

CodeReviewAgent

Direct LLM · cross-provider

Posts a GPT-4.1 review to the opened PR. Cross-provider by design — the reviewer is enforced to be a different model family than the generator.

How the metrics are computed

Agent pipeline — wall-clock from detected event to GitHub PR creation. Measured by scripts/measure_mttr.py against the live demo DB. Covers triage → diagnosis → fix generation → sandbox → PR open; excludes human review and CI time.
CI pass rate — fraction of generated patches that pass the Docker sandbox test suite on the first run. The sandbox clones the target repo, runs a baseline (pre-patch), applies the fix, and only fails the run if the fix introduces new test failures beyond the baseline.
Avg MTTR — wall-clock from detected event to merged fix PR, including human review and CI time. Measured across resolved incidents in the live demo DB. The ~6 min agent pipeline is the automated portion; the remainder is async human approval and CI.

Tech stack

Agent runtime	Anthropic SDK · Claude Opus 4.6 / Sonnet 4.6 / Haiku 4.5
Web framework	FastAPI · WebSocket streaming · Pydantic v2
Storage	Postgres (SQLAlchemy Core) · pgvector for RAG
RAG	Function-boundary chunking · hybrid search (α=0.7 vector + 0.3 lexical, min_score=0.45) · cross-encoder rerank · document chunk registry
Sandbox	Docker Compose · Jest · mongodb-memory-server
Tracing	Langfuse — every LLM call + tool execution as nested spans
Resilience	Circuit breakers · schema validation at handoffs · context checkpointing
Cost	Prompt caching (cache_control: ephemeral · 77% hit rate · 13% reduction on diagnosis) · state pruning (~18K tokens removed per FixGen run) · model routing Haiku / Sonnet / GPT-4.1
Deploy	AWS ECS Fargate · ALB · Cloudflare · GitHub Actions OIDC
Frontend	React + Vite · WebSocket dashboard · served from same container

Design decisions worth defending

Poll the log destination, not the server. filter_log_events against CloudWatch Logs every 5 minutes. Zero traffic to the production container — CloudWatch is AWS-managed storage; reading from it doesn't share capacity with real users. MTTD floor of 5 minutes; per-line log fidelity built in.
RAG finds candidates. The live store confirms truth. pgvector holds index-time snapshots; current incident state lives in Postgres. Blocking decisions always re-read the live store — the post explains why. Code retrieval uses hybrid search (70% vector + 30% lexical, min_score=0.45) with cross-encoder reranking — upgraded from pure vector after both agents were found to be passing low-relevance chunks to the LLM regardless of score.
Hard blocks are deterministic, soft hints are LLM-shaped. Dedup is a hard block (drop the event); regression context is a soft hint (prompt injection). Mixing the two created a four-failure-mode bug — walked through here.
Ground every symbol against the repo. DiagnosisAgent runs verify_symbol_in_repo via GitHub Code Search; a server-side guard re-checks every named function in the parsed output and rejects fabricated camelCase identifiers.
Sandbox before PR. Every fix runs in a Docker container against the real test suite. If tests fail, regenerate up to 3× before opening any GitHub noise.
Cross-provider review. Fix generation uses Claude Sonnet; code review uses GPT-4.1. Enforced at startup — the gateway raises on boot if both resolve to the same provider. Different model families have different blind spots; the reviewer is structurally incapable of being lenient with its own output.
Approval gate before every merge. All PRs require human approval; HIGH/CRITICAL incidents escalate immediately. Rejections are logged as RLHF preference pairs.
Two orthogonal controls on token cost. Prompt caching marks the stable harness prefix with cache_control: ephemeral — 77% cache hit rate, 13% cost reduction on the diagnosis step. State pruning removes stale read_file results from the FixGen conversation history every 3 iterations (~18K tokens per 10-iteration run, 29% reduction on that step). Neither interferes with the other: caching targets what is always the same, pruning targets what has already served its purpose.