Why My AI Agent Cited a File That Never Existed

Part 2 of the Agent Debugging series. Part 1 covered why the crash site is almost never the fix site. This post covers a subtler failure: a diagnosis agent that never got flagged as broken, because nothing about how it failed looked like a crash.

---

The card on the dashboard

It was a small thing. P3, the lowest severity we track. A Mongoose warning — "errors" is a reserved schema pathname — the kind of noise that shows up in logs and usually means nothing. DiagnosisAgent had triaged it, investigated it, and left a card sitting in awaiting_approval at 55% confidence, waiting for a human to glance at it before anything got shipped.

The card read well. Almost too well:

All four Mongoose model files use errors as a field name, which collides with Mongoose’s reserved schema pathname…

And under “affected file”: models/MasterBrandA.js, with a snippet showing exactly the offending block —

errors: {
  flagged: { type: Boolean, default: false },
  contentFlags: { type: [String], default: [] },
},

Specific. Plausible. Cited a real-sounding file. This is what a good diagnosis looks like from the outside — a clean, confident paragraph a tired on-call engineer would read once and approve.

Someone who actually knew the codebase looked at it and said: that’s not where the bug is. Not “I doubt it” — a flat correction. models/ValidationLog.js is the model that has the issue. Why is it looking for MasterModels?

That one sentence turned a routine approval into an investigation.

Pulling the thread

First move: don’t take anyone’s word for it, including the diagnosis’s. Pull the real file straight from GitHub.

models/MasterBrandA.js turned out to be a two-field email schema. No errors field. No flagged. No contentFlags. Nothing resembling the snippet the diagnosis had quoted, anywhere in the file. Checked every sibling Master*.js model too, in case the diagnosis just had the wrong one of several similar files. Same story in all of them. Searched the whole repo for that exact snippet, on the off chance it existed somewhere else and had just been mis-attributed.

It didn’t exist anywhere. The diagnosis hadn’t misread a file. It had written a file into existence, and quoted from it with total confidence.

The real bug was sitting in models/ValidationLog.js, which genuinely declares errors: { type: Array } — a one-line, unglamorous, completely correct root cause that the diagnosis never mentioned once.

To its credit, DiagnosisAgent never let this go anywhere dangerous. Confidence sat at 55%, below the bar to auto-generate a fix, and the incident sat waiting for a human. fix_attempted, fix_description, pr_url — all null. Nothing got shipped on the strength of a lie. But that’s a low bar. The paragraph a human reads before approving anything was still fiction, delivered with the same tone of voice as every diagnosis that’s actually right. A wrong answer that sounds unsure is a minor inconvenience. A wrong answer that sounds certain is a trap.

The sibling that didn’t fall for it

Here’s the part that made this worth chasing rather than just quietly correcting: the exact same warning came through again on a later run, low-confidence enough that it got routed to a different agent entirely — ErrorClarityAgent, whose whole job is adding a clarifying log line or error message when a diagnosis is too uncertain to safely fix outright.

This time it landed on the right file. Not almost-right. Not a plausible neighbor. The actual models/ValidationLog.js.

Same underlying bug, same shaky confidence situation, two completely different outcomes. That’s not a coin flip — something structural was different between these two agents, and it was worth understanding before writing this off as “the model just got unlucky once.”

DiagnosisAgent works the way most agents like it do: reason freely, call some tools, and when you’re ready, write a final answer — a root_cause_snippet, an affected_file, all in prose. It does have one grounding tool inside that loop, verify_symbol_in_repo, and the prompt makes calling it mandatory before naming a function. The model sees that result and can act on it in the same turn — if the tool comes back NOT_FOUND, the instructions are to null the field itself rather than guess. That part is interactive.

What it can’t check from inside the loop is the combination of its own claims: whether the function it named actually lives in the file it named, or whether the snippet it quoted as evidence is the file’s real, current content rather than something recalled from a similar bug elsewhere. Those checks run in a second pass, after the model’s answer is already final — ordinary Python re-reading the real file and testing for verbatim string containment. The model never sees the result of that second check. It doesn’t get a chance to say “oh, that’s not real, let me look again.” When it fails, the diagnosis prose itself isn’t rewritten, but the specific field it failed on is: affected_file, affected_function, root_cause_snippet all get reset to null, and confidence gets capped down along with them. The fabrication already happened; what’s left is stripping it out of the structured output and discounting whatever’s left.

ErrorClarityAgent doesn’t work that way at all, and not because someone specifically warned it not to hallucinate. It physically cannot commit a change unless the change is real. Its tool for proposing an edit — suggest_addition — requires a code_before string, and the code that actually writes anything checks, in plain Python, whether that string appears verbatim inside the real file before it will touch anything:

if a.code_before in new_content:
    # only now does anything get written

There’s no version of this where a fabricated code_before produces a commit. It’s not that ErrorClarityAgent is better at telling truth from fiction — it’s that its only way of acting runs through a needle that fiction can’t thread. The grounding isn’t a check bolted onto the process afterward. It’s the mechanism.

One agent checks its work after committing to an answer. The other can’t produce an answer that isn’t already checked. That’s the whole difference, and it explains the dashboard card perfectly without needing to blame the model for having a bad day.

Chasing the wrong culprits first

Before settling on that explanation, two other stories seemed just as likely, and both were worth actually checking rather than assuming.

Maybe the harness docs are poisoning it. The target app’s AGENTS.md documents a real per-brand pattern — a handful of white-label brand keys each map to their own per-brand model class (MasterBrandA, MasterBrandB, and so on) via a MODEL_MAP. It’s exactly the kind of memorable, structured fact a model reaches for when it’s underconstrained and needs something to say. Went looking for it: pulled 223 historical diagnosis cases from the eval dataset, 71 of them with real reasoning text, plus every live incident, and searched for any other diagnosis reaching for Master*/MODEL_MAP/per-brand framing on a bug that had nothing to do with it. Found exactly zero other occurrences. The one other historical case that even touched Mongoose — a strictQuery deprecation warning — stayed correctly scoped to the one real file involved. Whatever happened here, it wasn’t a standing bias baked into the prompt.

Maybe retrieval surfaced the wrong “similar” incident. DiagnosisAgent has a tool, search_similar_incidents, that pulls past incidents by similarity — if it grabbed some unrelated past incident about MasterBrandA and anchored on it, that would explain everything. Pulled the actual Langfuse trace for the run instead of guessing. That tool was never called. Not misused — never invoked at all.

Both plausible. Both wrong. Which meant the real answer was still sitting in the trace, unread.

What the trace actually said

Six observations, start to finish. One agent span, three model generations, and exactly two tool calls: check_still_occurring and get_error_samples. Both are the tools you’d reach for to answer “is this happening right now?” — nothing that reads a line of source code.

Both came back empty:

No ‘ERROR’ events in the last 120 min. Not occurring in the last 10 minutes.

And then the model answered anyway. Never called get_file_contents. Never called search_codebase. Never called grep_codebase. It had two negative log searches and nothing else — and when nothing else was available, it filled the space with the most structured, memorable fact sitting nearby: the MODEL_MAP pattern from the harness docs. Not malice, not a broken model — just a system that asked for an answer and didn’t require the answer to be built out of anything real.

Here’s the part that stung a little: there was already a safeguard for exactly this. An earlier version of this same bug — a diagnosis answering with zero tool calls at all — had already been caught and fixed, by requiring at least one real tool call before an answer would be accepted. That floor was sitting right there, doing its job. And this diagnosis walked right past it, because two calls to “is it happening” tools is still two tool calls. The floor counted calls. It never asked what they were calls to.

(One more thing turned up in that same trace, filed away rather than fixed on the spot: the model’s own raw output that turn contained a second, self-invented Observation: block — a fake tool result it wrote for itself, stitched on right after the real Action: line, describing a detailed log entry that was never returned by anything. The parser correctly recognized the real action and ran the real tool, ignoring the fake continuation for the purposes of execution — but the fake text still gets saved into the conversation history verbatim afterward, sitting there for the model to half-remember on a later turn. Not this bug’s mechanism, but the same instinct: when nothing real is available, invent something plausible and keep going.)

Closing the actual gap

The fix isn’t “tell the model not to make things up” — that instruction was arguably already implicit and it didn’t help. The fix is removing the option: don’t let two irrelevant tool calls satisfy a bar that was supposed to mean “you checked.”

BaseAgent picked up a second gate, sitting alongside the existing count-based one rather than replacing it — a named set of tools, empty by default so nothing else changes for any other agent:

self._required_tool_names_before_answer: set[str] = set()

If the set is non-empty, at least one call to one of those specific tools has to happen before an answer is accepted — not just any tool, that one. Miss it, and the model gets told exactly what it’s missing and sent back around:

REJECTED: you must call at least one of [‘get_file_contents’, ‘grep_codebase’, ‘search_codebase’] before answering — you have called [‘check_still_occurring’, ‘get_error_samples’] so far, which doesn’t include any of them.

DiagnosisAgent now sets that set to exactly the three tools that read real code. Log tools can still confirm something is happening. They just can’t be the whole story anymore.

Making sure it held

Five new tests, aimed squarely at the failure mode that mattered, not the easy cases: an answer offered after calling a tool but not the right one gets rejected (the actual production bug, reproduced directly); a correct tool call gets accepted immediately; an agent that never opts into the new set behaves exactly as before; an agent that keeps refusing to call the right tool eventually fails closed instead of sneaking an answer through on the last try; and when both gates are unmet at once, the model gets told about the one that actually matters.

Full suite ran clean against it — no new failures introduced anywhere else. Linting clean — the same discipline this whole investigation was trying to protect in the first place: don’t let something act just because it sounds ready to.

Left on the table

The tool-floor fix closes the specific hole this incident exposed. It doesn’t close the bigger asymmetry the investigation itself surfaced: DiagnosisAgent still checks its structured claims only after the model has already committed to an answer, in a pass the model never sees, with exactly one scripted retry if that pass fails. ErrorClarityAgent’s grounding is baked into the only door it has for acting at all — there’s no equivalent “after” for it to fail at. Giving DiagnosisAgent that same property, so a rejection is something the model can see and act on in the same conversation instead of a silent correction after the fact, is its own investigation — that’s next.

A few smaller threads, noted rather than chased down this time:

• That self-invented fake Observation: block is still there, still capable of contaminating a later turn’s context. Different bug, same instinct — noted, not yet closed.
• ErrorClarityAgent’s PRs — including the one that correctly fixed ValidationLog.js — never actually get reviewed by CodeReviewAgent. The review step is wired to DiagnosisAgent’s fix path and nowhere else. The agent that got it right isn’t getting a second set of eyes either.
• The original fabricated card is still sitting in the live incident store, awaiting_approval, with the fake root cause still visible to anyone who opens it. Safe to clear now — no PR was ever attached to it — but that’s a deliberate, confirmed action, not an automatic cleanup.