Engineering blog
Notes from building Remediate Labs — agent architecture, retrieval design, debugging stories, and the unglamorous decisions that make production agents actually work.
Series
All posts
We Built a Call Graph Because Our Agent Kept Breaking Callers It Never Knew About
The agent found one caller via GitHub search, patched it, and shipped. Two other callers broke in production. The fix was correct. The picture was incomplete.
Five Data Structures for a Call Graph — Why We Chose Hash Map of Sets
The adjacency matrix sounds clever. It's wrong for sparse graphs. The edge list is right for serialization and wrong for queries. Here's what each actually costs.
Cross-Encoder Re-Ranking — From Top-3 to Rank 1
The vocabulary gap query is at rank 3. Five functions have identical lexical scores — vector can't break the tie. Cross-encoder re-ranking sees query and document together and pushes the correct function to rank 1.
HyDE — Querying With Hypothetical Code Instead of the Error Message
HyDE pushed the code-vocabulary queries from 0.77 to 0.84. The incident-vocabulary query got worse: rank 3 → rank 5. The hypothetical went in the wrong direction.
Token Cost Engineering in Agent Loops — Prompt Caching and State Pruning
Output tokens cost 5× more than input. In agent loops, both compound. Two targeted fixes — prompt caching and state pruning — cut per-incident cost by ~25%.
Testing and Observability for Code RAG
Two ways to know your RAG is working: a recall harness you run before you ship, and chunk-level tracing that shows what actually retrieved in production. Neither replaces the other.
Alias-Based Deployment — Zero-Downtime Index Rebuilds
Without an alias swap, a full rebuild puts your index in a broken state for its entire duration. Queries return a mix of old and new results that never existed as a coherent snapshot.
When to Re-chunk, Re-index, and Re-embed
Three operations that sound similar. They are not. Each has different triggers, different costs, and different consequences if you get them wrong.
Why the Same Bug Kept Creating New Incidents (And What That Taught Me About RAG)
Three layers of dedup. Four independent failure modes that all had to fire simultaneously. The compound bug that exposed them, and the principle that makes it not happen again.
RAG Finds the Candidate. The Live Store Confirms the Truth.
A search index can't be a source of truth — that's a category error, not just a bug. The general rule for any vector index sitting in front of a live data store.
Why My AI Agent Kept Adding Null Checks Instead of Fixing the Bug
Five PRs to teach a fix-generation pipeline that the crash site is almost never the fix site. The producer/consumer distinction, RAG's structural blind spot, and what it took to find the actual bug.
Document Registry — Keeping the Index Honest
Upsert is insert-or-update by ID. It doesn't delete old chunks when a function is refactored. Without a registry, stale vectors accumulate silently until your LLM is reading code that was deleted six months ago.
Hybrid Search — Closing the Vocabulary Gap
The error query went from rank 7 to rank 3. The lexical bonus is 0.24 — larger than the 0.20 vector contribution. For this query, lexical search is doing most of the work.
Closing the Vocabulary Gap with LLM-Generated Descriptions
The error query went from not found to rank 7. Here's why rank 7 still isn't good enough — and what the score reveals about the trade-off between code vocabulary and incident vocabulary.
Fixing the Chunking Split with Function-Boundary Chunks
The function name query score jumps from 0.56 to 0.71. The natural language query goes from rank 3 to rank 1. Here's what changes when each function gets its own chunk.
Search Quality — Two Failure Modes and Why They're Different
One failure is a chunking problem. The other is a vocabulary gap that better chunking can't fix. Here's what the scores reveal about code RAG's limits.
What Actually Gets Indexed
A 1026-line file produces 26 chunks. Most boundaries fall mid-function, with no awareness of code structure. Here's exactly what gets lost — and why it matters for search quality.