# Combine searches without confusing rank with truth

A free, original LockedIn Labs lab. Allow about 40 minutes. Start with the visual, reproduce its calculations locally, then change one input and explain the result.

## 1. What hybrid search actually is

Hybrid search is a retrieval technique: ask more than one kind of search for useful material, then combine their results. It is not a language, a database or an agent. A search service can offer it; you can also build a small version in code.

Imagine two colleagues searching the same documentation. One looks for the words you typed. The other looks for related meaning. Each hands you a ranked reading list. You need a defensible way to combine those lists before deciding which passage supports an answer.

An FDE needs this distinction because business questions differ. An exact asset code may need a precise lookup. A customer’s paraphrase may benefit from semantic search. Neither gives a general guarantee of relevance, permission or correctness. Establish a simple baseline and test the failure you need to improve.

This lab uses four fictional account and billing passages from the [first embedding exercise](/training/guides/embeddings). P1 recommends an account-recovery page; P4 says not to use it. Both were marked current in that fixture. The contradiction is deliberate: the ranker cannot decide which policy is authoritative.

## 2. Follow the two lists

Choose a question in the visual below. The keyword lane counts distinct shared terms after removing a small, visible list of common words. It does not implement BM25, stemming or a production search engine. The semantic lane replays real cosine scores from the pinned CPU embedding exercise. The browser calculates keyword results and rank fusion; it does not run a model.

Reciprocal rank fusion, or RRF, gives each passage a contribution of `1 / (60 + rank)` from each list in which it appears. Rank starts at one. A passage missing from a list contributes zero for that list. Add those contributions, then order by the total. Equal totals use passage ID here, not an invented claim of authority.

The constant 60 smooths the influence of rank positions. It is separate from the number of candidates retrieved. In this lab it stays fixed while you vary the candidate limit. Scores from different methods have different scales, so do not add a keyword count directly to a cosine similarity. The resulting RRF score is also not an answer-confidence probability.

Before opening the third step, predict what changes when you keep only one candidate from each search. Then inspect the original source text. Did a conflicting instruction disappear from the candidate list, or was it actually resolved?

## 3. Run the local proof lab

Download `fde-hybrid-search.py` into a new working folder. Use Python 3.11 or later; no packages, model downloads, keys or network are needed. On systems where Python 3.11 is named `python3.11`, use that command instead of `python3` below.

```bash
python3 fde-hybrid-search.py test
python3 fde-hybrid-search.py inspect --query paraphrase --depth 2
```

The output has three lists: `keyword`, `semantic` and `fused`. Keyword entries include the actual matching terms. Semantic entries contain recorded model scores. Fused entries retain both ranks, both contributions and their sum. Trace P1: rank 1 in both lists produces `1/61 + 1/61`, approximately `0.032787`. Trace P4: rank 2 in both produces `1/62 + 1/62`, approximately `0.032258`. The program calculates before rounding.

The tests include a separate hand-checkable input: keyword order A, B and semantic order B, C. B wins because it appears in both lists; its total is `1/62 + 1/61`. These invented IDs test the algorithm, not a model or search-quality claim.

The semantic recording uses `sentence-transformers/all-MiniLM-L6-v2`, revision `1110a243fdf4706b3f48f1d95db1a4f5529b4d41`. It was calculated from full 384-value vectors in the existing embedding lab. This file reuses those outputs; it cannot infer scores for a new question. To change text or model, run the [embedding exercise](/training/guides/embeddings), record new results and evaluate them before comparing rankings.

## 4. Change the evidence before changing the model

Predict each result before running it. Keep one variable different in each comparison.

```bash
python3 fde-hybrid-search.py inspect --query paraphrase --depth 1
python3 fde-hybrid-search.py inspect --query paraphrase --depth 2
python3 fde-hybrid-search.py inspect --query paraphrase --depth 2 --omit-p1
python3 fde-hybrid-search.py inspect --query unrelated --depth 4
```

| Change | Inspect | What the result establishes |
| --- | --- | --- |
| Candidate limit 2 → 1 | P4 disappears for the paraphrase | A smaller candidate set hides the conflict; it does not settle the policy. |
| Leave P1 out before both searches | P4 rises; P1 contributes nowhere | Fusion cannot recover a source neither search supplied. A reranker can only reorder its input candidates. |
| Ask about tomorrow’s weather | Keyword list is empty; semantic search still ranks billing first | First place is relative to the collection. There is no weather evidence here. |

The omission switch models an incomplete candidate input. It is not a permission control. The original recording excluded a retired source and another organization’s source before embedding. In a service, enforce current access at the data boundary before giving text to a model. This public fixture contains no private data and proves no production access-control implementation.

To inspect all four recorded questions at all three limits, run:

```bash
python3 fde-hybrid-search.py compare
```

Do not call the twelve combinations twelve independent users, a benchmark or proof that hybrid search is better. They are controlled observations of a small teaching fixture.

## 5. Make a decision somebody else can review

Keep the query, candidate limit, omitted source, both input rankings and the fused result. Include a sentence explaining why raw scores cannot simply be added and a manual calculation for one passage.

Then write the evidence decision. For the password questions, identify the conflict and the policy owner or source version you would need to resolve it. For the weather question, say that relevant evidence is missing. The lab does not generate an answer or grant execution authority.

For your own system, define expected supporting sources independently before tuning. Include exact identifiers, paraphrases, contradiction, no-answer, stale-data and denied-source cases. Compare keyword-only, semantic-only and hybrid candidates on the same held-out questions. Track retrieval coverage separately from answer support, latency, cost and access failures. Add a reranker or graph retrieval only when a diagnosed failure and measured comparison justify it.

Use the [evaluation guide](/training/guides/evaluation-science) to design that comparison. The protected Academy lesson continues into choosing SQL, RAG, hybrid search or graph retrieval for a business question.

## Sources and next study

- [Microsoft: hybrid search overview](https://learn.microsoft.com/en-us/azure/search/hybrid-search-overview) — vendor implementation context; this local lab does not call Azure.
- [Microsoft: reciprocal rank fusion](https://learn.microsoft.com/en-us/azure/search/hybrid-search-ranking) — rank contributions and the distinction between fusion and later semantic reranking.
- [Sentence Transformers: all-MiniLM-L6-v2 model card](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) — the model used for the existing recording.
- [First embedding lab](/training/guides/embeddings) — reproduce actual inference and inspect the vectors before studying fusion.

Official retrieval documentation checked 23 October 2026. This is original independent instruction, not vendor certification or an affiliation claim.