"Email automation with LLMs: classification, drafting and the trust ladder"

Nobody wants an AI answering their email on day one, and they are right not to. An email sent under your name to a customer, a supplier, or a regulator is about the highest-stakes output a business automation can produce. It is also, handled properly, one of the highest-value things to automate: McKinsey Global Institute famously estimated that reading and answering email eats 28 per cent of the average knowledge worker's week, and nothing in the years since has made that figure feel exaggerated. The way to square that is to treat email automation not as one decision but as a ladder, and to climb it one rung at a time.

The bottom rung is classification, and it is worth far more than it sounds. The model reads each incoming message and assigns it to a fixed set of categories, enquiry, invoice, complaint, spam, action-needed, whatever fits your operation, plus a confidence score. It sends nothing. It writes nothing a human sees externally. It just labels and routes. This is the rung to start on because the cost of a mistake is tiny: a mislabelled email lands in the wrong folder and a human moves it. Meanwhile you get immediate value, triage, prioritisation, the right things reaching the right person, and, crucially, you start generating data about how good the model actually is on your mail, not on a benchmark.

The mechanics matter more than the prompt. Keep the category set small and mutually exclusive; ten fuzzy categories perform worse than five sharp ones. Always include an explicit "unsure" route, anything below a confidence threshold goes to a human-triage bucket rather than being forced into a category. Validate the output shape in code, because at volume the model will eventually return a category that does not exist. I run exactly this in my own business, on n8n, processing incoming email daily, and the unsure bucket is not a failure mode; it is the design working.

Once classification has run for a few weeks, you have the number that decides everything else: agreement rate per category. Not overall accuracy, per category. You will typically find the model is excellent at some categories and mediocre at others, and that distinction is what lets you climb safely rather than on faith.

The second rung is drafting for review. For categories where classification has proven reliable, the model now writes a reply, but a human reads, edits, and sends every one. The human stays the author; the model becomes the typist. Done well, this converts a ten-minute reply into a ninety-second review, and that is where most of the commercial value of email automation actually lives. Give the model the whole thread plus whatever context your systems hold about the sender, force a fixed structure and length, and never let a draft leave without human eyes. And measure the right thing: edit distance. If humans send drafts for one category nearly untouched and rewrite another category from scratch, that tells you precisely where the model is ready and where it is not.

The top rung is auto-send, and my honest advice is that most businesses should take years, not weeks, to put anything on it, and many never should. Where it is defensible is narrow, low-stakes, high-volume categories with a proven record: acknowledgements, receipt confirmations, standard information requests where the draft-review data shows humans changed essentially nothing across hundreds of cases. Even then: an allowlist of categories rather than a blocklist, a full audit trail of every sent message, sampled human review continuing forever, and a kill switch anyone on the team can pull without asking permission.

The ladder also works in reverse, and it needs to. When a model update shifts behaviour, or a new type of email starts arriving, the numbers slip, and a category that had earned auto-send should drop back to draft-for-review without drama. Rungs are earned with data and surrendered with data. Treating a promotion as permanent is how systems quietly rot.

What kills email automation projects is skipping rungs. Someone demos an end-to-end auto-reply, it works on ten hand-picked emails, it goes live, and within a fortnight it sends something embarrassing and the whole initiative, including the genuinely valuable classification and drafting layers, gets switched off. The demo proved the happy path exists. Only the ladder proves the system holds up when the input is strange and nobody is watching.

If you are looking at a swollen shared inbox and wondering whether AI can help, it almost certainly can, starting this month, at the classification rung. Building that ladder, with the measurement that lets you climb it safely, is the work I do. Get in touch via the contact form.