A confident answer is easy. A trustworthy one is harder to tell apart than you'd think.
An AI gives you a question, a confident final answer, and the reasoning trace it says it used to get there. Your one job each round: decide whether you trust the reasoning.
Some traces are sound. Others reach the same confident answer through a subtly broken step — a fabricated fact, an invented premise, a bad justification that happens to land right. The catch: a wrong reason for a right answer is exactly what's hard to notice. That's the thing this little study is about.
Trace shows the moment the answer does.
Runs entirely in this one file — offline, no accounts, no API key, no live model call. Every trace below is hand-written and fixed.
—
Do you trust how it got there?
In Seeing the Reasoning, people were shown model answers with reasoning traces and asked how much they trusted them. Changing how the trace was presented — shown instantly, held back, or offered on demand — barely moved trust. What moved trust was whether the reasoning was actually reliable. And when a trace was unfaithful — it reached the right answer through a fabricated or broken step — most people didn't catch it, and trusted it about as much as a sound one.
So a confident, fluent trace buys trust whether or not it earns it. The presentation you just picked is the part that doesn't matter much; the reliability underneath is the part that does — and it's the part we're worst at reading.
Why it matters: as more tools show their "chain of thought," a visible trace feels like proof of work. It isn't. Reading a trace critically — where's the load-bearing step, and is it actually true? — is a literacy skill, not a reflex.
Built as a lab experiment for whatupwolf.com/lab. One HTML file, no build step, no dependencies, no network calls — the traces are static and hand-written, so it works offline and reads the same on a plane. The finding it demonstrates is real; the example traces are illustrations of it.