Retrieval beyond cosine similarity

Vector search gets you 80% of the way in an afternoon and then stalls there for months. Cosine similarity finds documents that sound like the query. It does not find the documents that answer it.

Where naive retrieval breaks

Embed the query, embed the corpus, take the top-k nearest neighbours. It works until:

• Two chunks are near-duplicates and both win, wasting half your budget.
• The right answer uses different vocabulary than the question.
• A recency or permission constraint should have filtered a result out entirely.

The stack that actually works

1. Hybrid retrieval — combine dense vectors with a sparse keyword signal (BM25). Each catches what the other misses.
2. Metadata filtering — apply hard constraints before ranking, not after.
3. Reranking — a cross-encoder re-scores the shortlist by reading query and candidate together, which is far sharper than comparing two independent embeddings.

Similarity search is the cheap first pass. The quality lives in everything you do to the shortlist afterward.