Stop Constraining the Tagger. Let It Hallucinate, Then Correct It.

Every classification pipeline built on an LLM eventually runs into the same wall: the list of valid categories gets too long to put in the prompt. Simon Willison hit it on his own blog, which has accumulated 1,856 tags over two decades of posts — far too many to hand a model and ask "which of these fit this post?" in a single call.

The instinct most engineers reach for is retrieval: narrow the tag list first with a keyword or embedding search, then let the LLM pick from the shortlist. Doug Turnbull, in a post Willison linked to titled "Hypothetical Classifications", proposes something closer to the opposite.

Generate first, match second

Turnbull's method skips showing the model the taxonomy entirely. Instead:

1. Prompt an LLM to invent tags for a piece of content as if it were building a fresh taxonomy from scratch — with an example of the desired shape (e.g. a hierarchical path like Furniture / Living Room / Tables / Coffee) so the output is structured, not free-form prose.
2. Compute embeddings for those invented, non-existent tags.
3. Compare them against a precomputed embedding index of the real vocabulary and pick the closest matches by similarity.

In Turnbull's worked example, a model asked to tag "brown coffee table" invents a category like Furniture / Living Room / Tables / Coffee — which doesn't exist verbatim — and the embedding match resolves it to the real taxonomy entry, Furniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables. He implements the matching step with MiniLM embeddings and an in-memory set of vectors for the legitimate categories, and published the code as a notebook alongside his `cheat-at-search` repo.

Why the inversion works

The trick is that you're no longer asking the model to do a hard multiple-choice problem across thousands of options — a task LLMs are notoriously inconsistent at once the option count climbs. You're asking it to do the thing it's actually good at: produce a plausible, well-formed guess given a small example of the shape you want. The heavy lifting — narrowing thousands of possibilities down to one — gets handed off to a deterministic, cheap vector-similarity lookup instead of buried inside a giant prompt.

It also sidesteps a second problem: schema drift. Nothing about the vocabulary needs to be serialized into the prompt, so the taxonomy can grow or get renamed without touching the classification prompt at all — only the embedding index needs to be rebuilt.

The builder's takeaway

This is a small idea, but it generalizes past blog tags. Any agentic pipeline with a "route this to one of N buckets" step — support-ticket triage, product-catalog categorization, intent routing in a multi-agent system — hits the same wall once N gets into the hundreds or thousands. The pattern of *generate freely, then resolve by similarity* is cheaper to build than fine-tuning a classifier and more robust than trying to cram an ever-growing enum into a system prompt. It's a good instinct to keep in the toolbox for anyone shipping LLM-in-the-loop classification rather than a one-off blog-tagging hack.

Tell the model to output tags without any details of the existing vocabulary, then use vector embeddings against the existing corpus to find the concrete tags that are closest to the ones the model imagined might fit.

— Simon Willison, summarizing Doug Turnbull's technique