Two people get the same AI-drafted market analysis back. One skims it, likes the confident tone, and pastes it straight into a client deck. The other reads the same paragraph and stops on one sentence: a growth statistic that sounds plausible but doesn’t match anything they’ve seen elsewhere, phrased with exactly the same confidence as everything around it. They ask the model where the number came from. It can’t really say. That one check is the entire skill this post is about, and it has nothing to do with how good the prompt was.
Prompting gets most of the attention because it’s the part you can see yourself doing. Judging what comes back is quieter, easier to skip, and it’s where the actual risk lives — a well-crafted prompt can still produce an answer that’s fluent, confident, and wrong. This is a companion piece to how to write prompts that work: that post is about asking well, this one is about reading the answer well.
Three Different Things You’re Actually Judging
“Is this any good?” is really three separate questions, and conflating them is why a lot of AI output gets rubber-stamped or rejected for the wrong reason.
Product discernment asks about the output itself: is it factually accurate, does it actually meet what was asked for, is it coherent and appropriate for where it’s going. This is the check most people already do, at least loosely.
Process discernment asks about how the model got there, not just what it produced — and this is the one that catches problems the final text alone won’t show. Look for logical gaps, a detail the model fixated on while missing something more relevant sitting right next to it, or an idea you already rejected earlier in the conversation quietly resurfacing a few messages later as if it were new. A confident answer built on a shaky chain of reasoning can still read perfectly smoothly.
Performance discernment asks about the interaction itself, separate from any single output: is the model asking too many clarifying questions for a simple task, or too few for a genuinely ambiguous one, and does it actually adjust when you give it feedback, or does the next response ignore the correction entirely.
Most people only ever run the first check. The second and third are where the real judgment calls live.
Sycophancy: When the Answer Is Shaped by What You Want to Hear
Sycophancy is the specific failure product discernment is checking for: the model tells you what it thinks you want to hear instead of what’s actually true, agreeing with a factual error you’ve made, or quietly flipping its answer depending on how a question was phrased rather than on the underlying facts. It isn’t a bug that got missed — it’s a fairly direct side effect of training a model to be warm, agreeable, and supportive. Push a model hard enough toward “helpful and pleasant” and one of the failure modes you get back is a model that folds under pushback instead of holding a correct position.
Four things actually catch it in practice: phrase questions in neutral, fact-seeking language instead of leading language that signals what answer you’re hoping for; cross-reference anything load-bearing against a source you trust independently of the model; explicitly ask for the strongest counterargument or the case against what it just told you; and when something still feels off, step back and get a second opinion from an actual person rather than asking the same model to reassure you.
Where the Quirks Actually Come From
Sycophancy isn’t the only personality quirk that training produces, and knowing the mechanism makes the others easier to spot. A model’s character comes from two stacked processes: pre-training (predicting the next token across an enormous amount of text, which builds a broad statistical map of language) and fine-tuning (reinforcement learning from human feedback, which shapes the model toward specific behaviors people rated as helpful). That second stage is where a model’s “personality” actually gets set — and where unintended shadow-side traits ride along with the intended ones.
Two more worth watching for, alongside sycophancy: a verbosity default (three paragraphs when you asked for three bullet points), and being overcautious (a wall of caveats and disclaimers stacked onto a question that was never actually risky). Neither means the model is broken. It means the training process optimized for a general case that doesn’t match what you specifically need in this specific exchange, and pointing that out directly usually fixes it faster than rephrasing the whole prompt.
Knowing Where the Model’s Knowledge Actually Ends
Process discernment gets a lot easier once you have a mental model for where a model is reliable and where it isn’t. Think of it as a continuum rather than a flat “trustworthy” or “not”: topics that show up frequently and consistently across its training data, from before its knowledge cutoff, are where it’s genuinely deep. Move toward rare topics, very recent events, niche or local specifics, or less-represented languages, and reliability drops off — and unless a product has explicitly added a live search tool, there’s no real-time browsing filling that gap on its own.
The trigger conditions for hallucination map directly onto that same edge: specific statistics and citations, obscure or very recent topics, real-but-not-widely-known people or places, and exact dates, names, or numbers are exactly where confident-sounding errors are most likely. Three checks catch most of it: tell the model upfront that “I don’t know” is a completely fine answer, so it isn’t implicitly pressured to fill a gap with something plausible; open a fresh conversation and ask it to find errors in its own prior answer, which sidesteps it defending a position it’s already committed to; and apply extra scrutiny specifically to the fact types above, rather than treating the whole response as equally solid. None of this is a solved problem, and it isn’t presented as one — even Anthropic’s own training material frames hallucination as an ongoing, field-wide challenge, not a bug that got patched.
A Feedback Loop That Actually Corrects the Output
Spotting a problem is only half of discernment — the other half is fixing it without starting over. A short loop works better than a vague “this isn’t quite right”: name the specific problem, explain briefly why it’s a problem, give a concrete suggestion for what should change, and let the model revise against that. “This section contradicts the numbers in the intro, use the intro’s figures instead” corrects far more reliably than “can you check this again.”
Sometimes the loop itself reveals the real answer isn’t a better prompt at all — it’s that the task needed a person’s judgment from the start, and the AI was the wrong tool for this particular step regardless of how it was asked.
Why This Matters More Once You Publish or Ship Anything
Discernment done privately, in your own head, is optional. The moment AI-assisted content or output goes anywhere another person will see it — a client deliverable, a published article, a decision that affects someone else — the standard changes. You’re responsible for verifying it to the same level you’d apply if you’d written it yourself, not to whatever level felt sufficient while you were skimming it. That’s the same underlying argument behind platforms increasingly penalizing shallow, unverified AI content at scale: the discernment step is exactly the step that separates the two.
The Short Version
- Check the output, the reasoning, and the interaction separately — a fluent answer can still hide a reasoning error or a mismatched interaction style.
- Treat unexpected agreement with suspicion. If a model caves the moment you push back, that’s sycophancy, not confirmation you were right.
- Know the knowledge continuum. Specific stats, recent events, and obscure facts sit at the edge where confident-sounding errors are most likely.
- Correct with specifics, not vague dissatisfaction. Name the problem, explain why, suggest the fix.
- Verify before you ship it. Once AI-assisted output reaches someone else, the accuracy bar is the same as if you’d written it yourself from scratch.
Etienne Basson works with website systems, SEO-driven site architecture, and technical implementation. He writes practical guides on building, structuring, and optimizing websites for long-term growth.