Search & Retrieval

ANN algorithms — HNSW, IVF, ScaNN — find the closest vectors to a query without scanning all of them. They give up a small slice of recall in exchange for orders-of-magnitude speedup.

BM25 is a classical lexical retrieval algorithm that scores documents by how well their term frequencies match a query, with corrections for document length and rare-term importance.

Chunking is the process of splitting long documents into smaller passages that fit cleanly inside an embedding model's context window — and that align with semantic boundaries so each chunk is independently retrievable.

Dense retrieval finds documents by comparing their embeddings to a query embedding via cosine or dot product, served from an approximate-nearest-neighbor index.

FAISS (Facebook AI Similarity Search) is the C++ library for efficient similarity search and clustering of dense vectors. It implements the canonical ANN algorithms — flat, IVF, HNSW, PQ, and combinations — with CPU and GPU backends.

First-pass retrieval is the initial wide-net stage of a production search pipeline that surfaces a few hundred candidate documents per query out of millions. It optimizes for recall and speed; precision-at-the-top is left to a reranker downstream.

Grounded generation is the pattern of forcing an LLM's output to be derivable from a supplied set of retrieved sources, with citations attached. The standard defense against hallucination in RAG pipelines.

HNSW (Hierarchical Navigable Small World) is the dominant graph-based ANN algorithm. A multi-layer proximity graph supports log-time approximate search by greedy walks at each layer.

Hybrid search combines lexical retrieval (BM25) with dense retrieval (embeddings) into one ranked candidate set. Each method catches what the other misses, so the union is more recall-complete than either alone.

An inverted index maps each term to the list of documents (and positions) where it appears. The classical data structure behind keyword search — sub-millisecond lookups over billions of documents and the substrate every BM25 implementation builds on.

IVF (Inverted File Index) is the cluster-based ANN algorithm: K-means partitions the corpus into a few thousand cells, each query is matched to its nearest centroids, then exhaustively searched within only those cells.

Parent-document retrieval splits the index granularity from the context granularity: embed and retrieve over small chunks for precision, but return the larger parent document to the LLM. Fixes the chunk-boundary problem in RAG.

Product quantization (PQ) compresses a vector by splitting it into M sub-vectors and quantizing each independently against a small codebook learned via K-means.

Augmenting the original query with synonyms, paraphrases, or hypothetical answers before retrieval. The classical IR technique that LLMs reinvented as HyDE. Sometimes a clean win, sometimes drift that hurts more than it helps.

Query rewriting transforms a user's raw query into one or more reformulated versions tuned for retrieval — expanding abbreviations, decomposing multi-part questions, or fixing the syntax expected by an underlying search API.

RAG is the pattern of retrieving relevant documents and feeding them into an LLM as context, so the LLM can answer with grounded, citeable information instead of guessing from its training data.

Reciprocal rank fusion (RRF) is the boring, parameter-free way to merge multiple ranked lists into one. Sum $1/(k + \text{rank})$ across lists with $k=60$ — and you have the default fusion method in production hybrid-search stacks.

Semantic search is the umbrella term for retrieval that goes beyond surface keyword matching to capture meaning — most often via dense embeddings, but also via learned-sparse models, query rewriting, and reranking.

Sparse retrieval is the family of methods that represent queries and documents as high-dimensional sparse vectors over a vocabulary — including BM25 and modern learned-sparse models like SPLADE and uniCOIL.

SPLADE (SParse Lexical AnD Expansion) is a learned sparse retrieval model: a transformer produces a sparse term-weight vector over the BERT vocabulary for each query and document, scored by dot product on an inverted index.

TF-IDF weighs a term by how often it appears in a document (term frequency) times how rare it is across the corpus (inverse document frequency).