RAGLensGlossary

Glossary

The retrieval and evaluation terms used across RAGLens. Plain English first; the precise version underneath for anyone who wants it.

Research: structure-aware retrieval →

Retrieval: Finding the pieces of source content that might answer a question, before any answer is written. Everything downstream depends on what retrieval finds.; PreciselyIn a RAG system: selecting a subset of indexed content (chunks, units, documents) for a query, by vector similarity, filters, graph traversal, or a combination.
Semantic search: Search by meaning rather than exact words. A question about "broken mugs" can find a page titled "damage claims" because the meanings are close.; PreciselyDense vector retrieval: the query and each piece of content are embedded into vectors, and results are ranked by vector similarity (here: cosine over 1024-dimension Voyage-3 embeddings).
Embedding: A list of numbers that represents what a piece of text means. Texts with similar meanings get similar numbers, which is what makes semantic search work.; PreciselyA learned vector representation of text. This site uses voyage-3 (1024 dimensions); similarity is computed between query and content vectors.
Metadata: Facts about a piece of content rather than the content itself — which product line it covers, which year it applies to, when it was last reviewed.; PreciselyStructured fields attached to indexed content that retrieval can filter or rank on (e.g. an applies_to version field used in a Pinecone metadata filter).
Applicability: Whether a piece of content actually applies to the situation in the question. A 2025 fee policy is real content — it just doesn’t apply to a 2026 order.; PreciselyIn the experiment: a package-level metric scoring 1 when no forbidden (inapplicable, e.g. superseded-version) unit is present, 0 otherwise.
Knowledge graph: Content plus explicitly authored relationships between pieces of content: this task requires that step first, this rule has that exception, this policy replaces that one.; PreciselyHere: typed directed edges (requires, refines, supersedes, related) over content units, traversed deterministically at retrieval time — no graph database, no LLM-inferred edges.
Prerequisite: Something you must do or know before the thing you asked about. An answer that skips a required prerequisite is incomplete even if everything it says is true.; PreciselyModeled as a requires edge from a task unit to the unit it depends on; the graph condition follows these edges one hop from retrieved tasks.
Exception: A rule that changes another rule in specific cases. "Rush fees are waived when the delay is the supplier’s fault" only matters if you see it next to the rush-fee rule.; PreciselyModeled as a refines edge from the exception unit to its base policy; the graph condition follows refines edges in both directions from retrieved policies.
Supersession: One document replacing another. The old version usually still exists — and often still reads as a perfectly good answer.; PreciselyModeled as a supersedes edge between versioned units. The graph condition swaps a retrieved superseded unit for its applicable successor; the metadata condition removes inapplicable versions by filter.
Related link: A loose "see also" connection between two pieces of content — weaker than a prerequisite, exception, or replacement.; PreciselyThe related edge type. In the experiment it was isolated in a stress arm (Graph + related), which measurably increased irrelevant evidence and never improved recall.
Evidence: The retrieved content an answer is allowed to draw on. If it’s not in the evidence, a grounded answer shouldn’t say it.; PreciselyThe packaged set of content units passed to generation for one question — what the experiment scores for recall, irrelevance, and applicability.
Evidence limit: A cap on how much retrieved content an answer can use. Every retrieval approach in the experiment worked under the same cap, so none could win by simply bringing more.; PreciselyA shared token budget (1,400 tokens, calibrated) applied to every condition’s evidence package; packaging rules never truncate a unit mid-text.
Evidence recall: Of the evidence a correct answer needs, how much did retrieval actually find? Missing required evidence is the quiet way answers go wrong.; PreciselyRequirement-level recall: |satisfied requirements| / |requirements|, where an any-of requirement is satisfied by any of its listed units.
Seed: One of the initial search results an approach starts from. Graph-based approaches expand outward from their seeds — so everything depends on what got seeded.; PreciselyThe top-S applicable dense results (S = 4 for the graph conditions here, vs. a flat top-7 window for the non-graph conditions) from which typed expansion proceeds.
Seed starvation: When the initial search results all point the same (wrong) way, and no authored relationship leads from them to the evidence that’s actually needed — so the graph never finds it.; PreciselyFailure mode observed on the partial-answer questions: required units neither seeded within S = 4 nor reachable by any typed edge from any seed, yielding 0.00 requirement recall for the graph condition.
Provenance: Why each piece of evidence is there: found by search, pulled in as a prerequisite, swapped in as a replacement. It turns retrieval from a black box into an explanation.; PreciselyA machine-readable reason attached to every unit in a package (dense match, prerequisite-of, exception-of, supersession replacement, section context, related link).
Abstention: Declining to answer because the content doesn’t support one. When the source material is silent, "I can’t answer that from the handbook" is the correct, healthy outcome — not a failure.; PreciselyA judged boolean on each generated answer; on unanswerable questions, every condition in the registered run abstained in all repetitions.
Unsupported claim: A statement in an answer that the source content never makes. The most dangerous ones sound like perfectly reasonable policy.; PreciselyIn the experiment: an unanswerable aspect judged asserted — a fabrication event. Aspect-specific boolean checks caught occurrences that generic faithfulness metrics did not reliably flag.
Grounded answer: An answer built only from the retrieved evidence, saying so plainly when the evidence runs out.; PreciselyThe generation contract for every condition and the live site: answer from the provided context only; acknowledge gaps rather than filling them.
LLM-as-judge: Using a language model to grade answers — fast and scalable, but a judgment, not a measurement. Good evaluations say exactly what the judge was asked and what it was allowed to see.; PreciselyHere: a blinded judge scores answers without seeing condition identity, similarity scores, or provenance; completeness is judged per key fact and per unanswerable aspect as explicit booleans at temperature 0.
Registered experiment: An experiment whose predictions, measurements, and decision rules were written down and frozen before it ran — so the results can’t be quietly reshaped after the fact. Negative results get published.; PreciselyDesign note committed before implementation; gold set frozen at a recorded commit; the registered run cites that freeze; later changes void registered status. The decision applied is the preregistered rule.