"Your retrieval is the problem, not your model"
When a RAG system gives a bad answer, the instinct is to blame the model and reach for a bigger one. Most of the time the model was never shown the right material. One debugging habit — read what was retrieved before judging what was generated — resolves the majority of cases.
A bad answer comes out of your RAG system and the team's first instinct is to blame the model. Swap it for the bigger one, rewrite the prompt, add "you must answer only from the provided context" in increasingly stern language. I understand the instinct — the model produced the words, so the model gets the blame — but in my experience the majority of bad RAG answers have nothing to do with generation. The model was never shown the right material, and no model answers well from the wrong page.
Have an AI feature stuck between demo and production?
The gap — reliability, evals, cost control, the plumbing that keeps it running unattended — is exactly the work I do. If that sounds familiar, a short conversation is usually enough to point you the right way.
Book a free consultation