About
What RAGLens is
RAGLens is a compact retrieval-augmented answering pipeline wrapped in a diagnosis layer. Most RAG demos show an answer and a similarity score. RAGLens helps distinguish whether supporting evidence was retrievable at all, whether retrieval surfaced it, and whether the answer used it — or went beyond it.
Why diagnosis matters
Ask the Google Developer Style Guide corpus: “How should I write alt text for images in documentation?” It sounds like exactly the kind of question a style guide answers. In a live run, retrieval scored 0.15 and the answer scored 0.95. A single aggregate score would hide that difference.
The two scores measure different things. Retrieval scored low because the indexed corpus did not provide retrievable support: an exhaustive pass over every indexed chunk found nothing above the calibrated relevance threshold. The answer scored high because it did the right thing with that situation — it said the sources don’t cover alt text instead of inventing guidance.
That suggests a possible content gap, but retrieval alone cannot confirm one. An exhaustive retrieval pass proves only that this retriever found nothing; the content may exist in a form this retriever cannot match. Confirming a gap is the corpus owner’s call, so RAGLens flags the case for corpus review rather than declaring the corpus incomplete.
When refusing is the right answer
Language models are not deterministic, and the same question does not always get the same behavior. Sometimes the model declines cleanly — a correct refusal, scored as healthy behavior, not failure. Sometimes it answers anyway, producing plausible-sounding alt-text guidance from its training data with nothing retrievable behind it.
The evidence is the same in both runs; the difference is whether the model abstains or answers without support. RAGLens distinguishes those outcomes: the refusal is labeled a correct abstention, and the confident answer is labeled answered without retrievable support.
How the diagnosis works
Retrieve evidence
Dense vector search pulls the top paragraph chunks into the generation window. A deterministic broad pass — exhaustive on these corpora — records what exists beyond it.
Generate answer
One generation call, grounded on the retrieved window only. The pipeline is deliberately compact; the same evaluation questions can also be applied to hybrid and agentic stacks.
Judge relevance + evidence
An LLM judge scores the retrieved window only: relevance, evidence sufficiency, answer groundedness, and whether the answer correctly abstained.
Diagnose outcome
Judge scores combine with broad-pass availability into a failure label, and every verdict states its coverage: exhaustive, broad pass, or window-only.
Relevance thresholds are calibrated per corpus against human-labeled test cases. A golden evaluation set is not required for every diagnosis: any question can be run, while labeled cases are used to calibrate and sharpen the system. The same model family currently generates and judges, which introduces self-evaluation bias. A cross-model judge is future work.
How I work
I built RAGLens as a product manager working with AI collaborators across research, information architecture, and development. I set the questions, research and product decisions, and interpretation; the AI team helps inspect source material, implement conditions, run tests, audit artifacts, and challenge conclusions.
Research starts with a specific problem or claim worth investigating. Some questions become controlled experiments. Others stop at an audit or decision note when the evidence is already enough to make a decision.
See it live
Run the alt-text example from the Evaluate one query page — it’s the “Evidence not found” card under the Developer Style Guide corpus. Or run Evaluate in bulk, where curated cases — including expected refusals — run against the live pipeline.
Browse Research for retrieval-design experiments, decision notes, and the questions they lead to next.