Reranking
A second-stage scoring of retrieved chunks using a stronger model to improve RAG precision.
What is Reranking?
Vector search is fast but imprecise — it finds chunks that are roughly relevant. Reranking takes the top-N candidates from vector search and scores them more carefully using a cross-encoder model (Cohere Rerank, BGE Reranker, etc.) that reads the query and chunk together.
The typical pipeline: vector search retrieves top-100, reranker scores those, top-5 are sent to the LLM. The reranker is slower per pair (it reads the query each time) but only runs on 100 candidates — total latency stays acceptable.
Reranking is the single most impactful RAG improvement after good chunking. It often closes the gap between "demo-quality" and "production-quality" answers.
Reranking is what separates RAG systems that work in demos from RAG systems that work in production. Every senior Gen AI engineer adds it.
A Bengaluru customer-support RAG was confidently returning irrelevant policy pages. Adding Cohere Rerank between vector retrieval and the LLM cut wrong-citation errors from 18% to 3% — without changing the underlying embeddings.
Want to master this?
Learn Reranking in a structured cohort
3-month live program with mentors, real projects, and 50+ partner placement support.