The AI Double Trap

Here is the claim, stated so you can disagree with it in one sentence: pointing a capable model at your pipeline makes your judgment worse before it makes it better, and the fix is structural, not a smarter model. If your plan is to wait for the next release and let it read your deals, I think that plan is wrong – and I want to say exactly where, rather than wave at it.

I've written about this at a higher level in the thesis. This page is the mechanics: what the trap actually looks like inside a pipeline, the strongest case against what I'm claiming, and what I think genuinely escapes it.

Two traps, and they are not the same failure

The name suggests one problem. It's two, and they catch different people – you, and the model itself.

Trap one: you get trapped, because the output looks like progress. A model reads a stalled thread and hands you a clean summary: champion engaged, budget confirmed, next step is a demo. It reads well. Nothing about it looks unfinished. That's the problem – the fluency is exactly what stops you from checking it. Ask around and you find the champion left the company three weeks ago, and "budget confirmed" was one line from a call four months back that nobody revisited since. None of the individual facts in the summary are invented. They're real fragments, arranged into a coherent story that stopped being true a while ago. A rough bullet list would have prompted a "wait, is this current?" A well-written paragraph doesn't, because well-written is the whole tell you've learned to trust.

Trap two: the model gets trapped, because it tunnels. Ask it to rank forty deals by likelihood to close this quarter and it will rank on whatever you handed it prominence for: recency of contact, deal size, stage. It will not, on its own, notice that three of your top five ranked deals share a champion who just changed jobs – a fact sitting in email headers, not in the deal notes you fed it. This isn't a knowledge gap. Ask the same model directly whether a champion's job change matters to a live deal and it will tell you, correctly, that it matters a great deal. But once it's executing the ranking task, that's the frame, and nothing pulls it back out to ask what it isn't looking at. The tunnel isn't ignorance. It's commitment to the track you put it on.

Put together: trap one is about you trusting a finished-looking answer. Trap two is about the model never widening the lens on its own. You can fall into either one without the other – a rough, obviously-incomplete ranking can still tunnel; a confident summary can still be scoped correctly. Pipeline work usually gets both at once, because the same instinct that makes a model good at looking finished is the instinct that makes it commit hard to a frame.

The strongest case against this

I'd rather state the counter-argument the way someone smart would make it than build a version I can beat.

Reasoning models now show their own uncertainty – caveats, confidence markers, "I'm inferring this from indirect signals." You can prompt around the tunnel directly: tell it to check for champion changes, ownership transitions, anything stale, and it will. And every model generation this cycle has been meaningfully better than the last at exactly this kind of judgment call, so betting that the gap stays open reads like betting against a trend that keeps proving people wrong.

All three are true, and I'll concede the honest part of each. Reasoning traces do make individual outputs better calibrated. Prompting for a known failure mode does fix that failure mode. And yes, each generation is better than the one before it at reading a messy paragraph correctly.

Here's what none of that touches: you can only prompt for the consideration you already know to ask about. The failure that costs you a deal isn't the one that's slightly miscalibrated inside the frame – it's the one that never entered the frame at all, and a caveat only gets attached to something the model chose to say. A better model raises the average quality of every individual read. It doesn't add a step that asks "what did this run not consider," because that question has to come from outside the run, not from inside it. Nothing in a single pass checks for its own absence. That's a structural gap, not a capability gap, and no amount of the model getting smarter closes a gap that sits outside where the model is looking.

What actually escapes it

Not a better prompt, and not a bigger model. What works is putting enough outside the model call that its judgment stays checkable rather than final: a framework that defines what matters for your specific pipeline before the model ever runs, so "what did it miss" has a fixed list to be checked against; checkpoints that force a second, independently-scoped pass instead of trusting the first one; and a human who stays the last decision-maker on anything touching a live relationship, not a rubber stamp on what the model already decided.

None of that is exotic, and none of it is secret – it's mostly the unglamorous discipline of not letting one model call be the whole loop.

What would change my mind: an architecture where re-running a ranking cold, on the same pipeline, with no prompt change and no new fact supplied, reliably surfaces a consideration the first run missed. I haven't seen that yet. If you have, I'd like to see it.