Thinkmill’s October 27, 2026 guide, “AI search explained: indexing, retrieval and reranking,” compares three retrieval approaches on the same synthetic dataset. Its useful lesson for teams building retrieval-augmented generation, or RAG, is that finding relevant documents and putting the best ones first are separate jobs.
Author Dinesh Pandiyan uses a bike company’s scattered records to illustrate the difference. A product manager wants to know why customers are requesting refunds. Keyword search finds pages containing “refund,” including policy and processing instructions. Semantic search can connect that question to complaints written in different language. A reranker then evaluates the shortlisted documents against the question, promoting evidence about the reasons for returns.
The practical decision is where to spend the next unit of engineering effort. If useful evidence never reaches the shortlist, improve retrieval. If the evidence arrives but sits below less useful pages, reranking is a more direct intervention.
Ten Notes Separate Topic Matches From Answers
Thinkmill’s dataset contains ten short notes from eight sources, including a wiki, support desk, email, Slack and a bug tracker. Three notes answer the refund question:
- A support ticket says a rear brake cable snapped on the second ride and the customer is returning the bike.
- An email describes a refund request because the battery lasts 10 kilometers, against an advertised 60 kilometers.
- A Slack message reports three customers wanting their money back because of faulty brakes.
Only the email contains the word “refund.” Two other documents contain that word but answer different questions: what the refund policy permits and how to process a refund.
That distinction makes the example useful. A search result can match the subject perfectly while failing to answer the question. The policy page belongs to the topic of refunds; the broken-brake complaint supplies a cause.
I’d use this distinction when creating a retrieval evaluation set. Labeling every refund-related document as relevant would hide the exact failure the guide exposes. For this question, relevance means evidence explaining customer behavior, not simply a shared subject.
The dataset is synthetic and small. It illustrates architecture, but it cannot establish a production relevance gain, latency figure or cost saving for a company’s own documents.
What Each Search Approach Actually Matches
Keyword Search Matches Indexed Terms
Text search prepares documents ahead of time by analyzing their words and building an index. Thinkmill describes processing such as tokenization, lowercasing and removing common stop words.
At query time, the engine looks for indexed terms and ranks matching records. A literal search for “refund” has an obvious weakness in this dataset: “money back” and “returning the bike” express related intentions without using that term.
Calling keyword search an exact-word matcher is a useful introduction, but production lexical search has more controls. Analyzers and query configuration affect matching. Microsoft’s RAG retrieval documentation describes matching any or all query terms and searching fields such as titles, summaries, keywords and entities.
Those controls matter when exact names or identifiers carry the answer. A product code, error message or policy identifier may deserve a strong lexical match even when surrounding documents look semantically similar.
My view is that keyword search remains a valuable retrieval channel. Thinkmill’s example shows its vulnerability to vocabulary differences, not a reason to remove it from every search stack.
Embeddings Match Representations of Meaning
Embedding search converts document text into numerical vectors during indexing. At query time, an embedding model converts the question into a vector, and the search system finds nearby document vectors using a similarity measure.
This lets a refund question match language about returning a bike or wanting money back. The words differ, but their representations can be close enough for retrieval.
Microsoft documents cosine similarity, dot product and Euclidean distance as supported similarity measures in Azure AI Search. It also specifies that query embeddings must use the same embedding model as the document embeddings. The configured vector dimensions must match that model’s output.
Similarity still differs from answer usefulness. A refund policy and a refund complaint can both sit near a refund question in vector space. Embedding retrieval can bridge the vocabulary gap while leaving the final ordering imperfect.
The operational benefit comes from doing much of the document work ahead of time. Stored document vectors can be searched repeatedly; each new query needs its own embedding and a search against the index.
Reranking Spends More Work on Fewer Documents
A reranker adds a second scoring stage after initial retrieval. Thinkmill’s guide describes a model scoring the query against each shortlisted document, then changing their order.
A common implementation is a cross-encoder, which reads the question and a candidate passage together. Abhyuday Desai’s Retrieval Explained series uses this approach to demonstrate moving an answer-bearing passage higher in a shortlist.
That joint assessment gives the model a more specific task than finding nearby vectors: judge how well this particular passage addresses this particular question.
In Thinkmill’s example, that means pushing documents explaining refund causes above policy and how-to pages. For RAG, the consequence is straightforward. If an application passes only a few retrieved passages to its language model, their order determines which evidence survives the cutoff.
The extra stage also has a cost:
| Approach | Main matching signal | Work prepared during indexing | Work added at query time |
|---|---|---|---|
| Keyword search | Indexed terms and configured text matching | Analyze text and build the index | Match and rank indexed terms |
| Embedding search | Similarity between query and document vectors | Generate and store document embeddings | Embed the query and search vectors |
| Embedding search with reranking | Vector similarity, followed by query-document relevance scoring | Same embedding preparation | Search, then score shortlisted query-document pairs |
This is an architectural comparison, not a price table. Actual expense depends on the models, hosting or API arrangement, document lengths, query volume and candidate count.
Scoring more candidates generally requires more reranking work. Longer passages also give the model more text to process. Batching and hardware affect latency, so candidate count alone cannot predict response time.
I’d justify that additional stage through a user-facing outcome: better evidence in the final context or more successful search sessions. A cleaner-looking ranking is useful only if it improves what the application delivers.
The Shortlist Sets a Hard Ceiling
A reranker can reorder its inputs. It cannot retrieve a document it never receives.
That limit is the most consequential part of Thinkmill’s guide because it separates two failure modes that can produce the same bad answer:
- Retrieval failure: the relevant passage is missing from the candidate list.
- Ranking failure: the passage is present but falls below the application’s final cutoff.
The fixes differ. A stronger reranker may address the second problem. The first requires changes upstream, such as broader retrieval, a different search method, better query formulation or improved document preparation.
Consider a hypothetical extension of the guide’s example. There are three known relevant notes, but the initial search retrieves only two. Candidate recall is therefore 2 ÷ 3, or about 67%. Even a perfect reranker cannot place the missing third note into the final context.
That arithmetic produces a useful diagnostic rule: measure candidate recall before judging the reranker. Otherwise, a team can spend time tuning the ordering of a list that lacks the necessary evidence.
Frequently Asked Questions
4 questions
1Can a Reranker Find Documents Search Missed?
No. A reranker scores the candidate documents it receives. Missing evidence must be recovered through changes to retrieval or document preparation.
