"Chunking is a product decision, not a preprocessing step"
Chunk size is usually set once, by a default, and never revisited — yet it determines which questions your RAG system can answer at all. Chunking decides what the model is allowed to see, which makes it a product decision wearing a preprocessing costume.
Ask your document assistant a question whose answer is a whole section, and watch it respond with a fragment. Ask it for one specific figure, and watch it drown the figure in three paragraphs of adjacent noise. Both failures trace back to the same line of code: the one where somebody set a chunk size to 512 tokens because that was the default, and never thought about it again.
Have an AI feature stuck between demo and production?
The gap — reliability, evals, cost control, the plumbing that keeps it running unattended — is exactly the work I do. If that sounds familiar, a short conversation is usually enough to point you the right way.
Book a free consultation