Claude 3.5 is smarter than Claude 3. GPT-4 is smarter than GPT-3.5. Neither is immune to hallucination.
I've run identical prompts across Claude, GPT-4, Gemini, and Bedrock-hosted models side by side. The bigger models don't hallucinate less so much as they hallucinate differently — they're more confident about it. Claude hallucinates less often, but when it does, it's harder to catch, because the failure is smooth rather than obviously wrong. GPT-4 hallucinates more visibly — you can usually spot the seams. Early Gemini versions hallucinated constantly and it showed.
The gap between models is real. It's also small compared to the gap between a system that mitigates hallucination and one that doesn't.
What I've seen in production
At BCG, I worked a project where the brief was "upgrade from GPT-3.5 to GPT-4." We did that. We also added three things on top: citation checking, confidence thresholding, and explicit fallback strategies. The model upgrade helped. The mitigation work helped considerably more. The client's feedback made that obvious — they noticed the mitigation, not the model swap.
This is an inconvenient fact for the industry, because model vendors — Anthropic included — make their money selling bigger models. The implied upgrade path is always "buy the next model." For some workloads, that's genuinely the right call. For most production systems I've reviewed, it's backwards.
The right order
The order should be:
- Make the system verify its own output — citations, confidence checks, fallback chains.
- Then optimize the model, if you still need to.
I see this inverted in something like 80% of the systems I review. Someone gets a Claude or GPT-4 API key, wires up a quick RAG pipeline, and assumes the model handles hallucination on its own. It doesn't. The system does — or it doesn't exist yet, and that's the actual gap worth closing.
Related reading: RAG systems still hallucinate. Here's what actually stops it.