Research
Practical investigations into retrieval, evidence, and AI answer quality.
Retrieval design
A diagnostic approach to deciding what to change in a retrieval system, and when.
Add complexity only when an observed failure mode warrants it.
Content system
WixQA
A sequence of studies using SaaS help content to examine retrieval breadth, applicability, and representation.
Retrieval breadth · Experiment
How much should retrieval retrieve?
Outcome: Increasing retrieval breadth improved recall quickly at first. By k=20, the gains had started to flatten.
Read experiment →Applicability filtering · Decision note
Would filtering address the retrieval failures left after increasing retrieval depth?
Outcome: No. Wrong-scope competition explains too few of the remaining failures. Investigate representation instead.
Read decision note →Structural chunking · Experiment
Does chunking help when whole articles fail?
Outcome: Sections recovered 24 previously missed sources and lost 19 previously found ones. Net gain: five sources in the top five. Chunking changed the ranking far more than it improved retrieval.
Read experiment →
Content system
Supplier handbook
A small fictional handbook with versioned content, explicit applicability metadata, and authored relationships between documents.
Structure-aware retrieval · Experiment
What happens when we add metadata filtering and a knowledge graph?
Outcome: Keep the version filter. Rethink the graph approach.
Read experiment →Seed breadth vs. a second search · Experiment
When retrieval misses evidence, is it better to search more broadly or ask the model to reformulate the search?
Outcome: Tune the obvious lever first. The extra model call did not earn its cost.
Read experiment →