Retrieval Systems

Why

Retrieval connects generation to changing, private, or domain-specific knowledge. Its real purpose is not to find text; it is to assemble the smallest evidence set that supports the next decision.

Mental Model

Treat retrieval as a funnel: corpus → candidates → ranked evidence → assembled context → grounded answer. Every stage can lose signal or add noise.

Core Concepts

• Chunk boundaries and document structure
• Sparse, dense, and hybrid search
• Metadata filtering and access control
• Reranking and diversity
• Context packing and citation mapping
• Freshness, deletion, and index lineage

How It Works

Documents are normalized, segmented, enriched with metadata, and indexed. A query is transformed into one or more search requests. Candidates are filtered and ranked, then selected under a context budget. The model receives the evidence with instructions that define how it may use it.

In Harness

The trace stores query transformations, candidate identifiers, scores, filters, selected passages, and their positions in the final context. Retrieval failures can then be separated from generation failures.

Engineering

Preserve source identifiers through every transformation. Enforce access before ranking. Evaluate retrieval independently with recall and ranking metrics, then evaluate the full task. Design deletion as a first-class pipeline operation.

Trade-offs

Smaller chunks improve targeting but lose context. Hybrid retrieval increases coverage but adds tuning. Reranking improves precision but consumes latency. Larger evidence sets can improve recall while reducing answer focus.

Common Mistakes

• Tuning only on easy keyword queries.
• Embedding tables and headings without structural context.
• Using similarity score as calibrated confidence.
• Omitting empty-result and conflicting-source behavior.

Practice

Create a 30-query set containing direct, paraphrased, multi-hop, negative, and stale-information cases. Inspect the top ten candidates before changing the generator.

Sources

See the reference library for information-retrieval and vector-search material.