On this page
Most questions about work are not questions about one document. Someone asking who will sign off on a release and when the dry run begins needs information that may be split between an email thread and a calendar entry. An answer that finds only one looks relevant but is incomplete.
Zortex is being built around a private model of a person's work, assembled from connected apps and made available to an AI with the permissions the person chooses. The first measurable task is evidence completeness: did retrieval collect every record the question depends on?
Dataset and run: The October 9, 2026 run uses generated project records and a deterministic ranker. The numbers measure retrieved record IDs.
Question
For a fixed collection of generated work questions, how does the amount of retrieved context affect the fraction of questions for which all required evidence is present? And where does adding more retrieved text stop helping this particular task?
The metric is evidence coverage: whether the required source records are present in the retrieved set. A model can misunderstand complete source material or omit a decisive qualification. Conversely, a person might answer from knowledge not present in the selected records. Our test has none of those behaviors.
Setup
The generator makes 24 projects with identifiers such as arc07. Each project has five short records: a sign-off email, a dry-run calendar note, a design update, a standup note, and an older meeting archive. The last three are intentionally plausible distractors. Each project also gets three alternate formulations of a question requiring the sign-off reviewer and the dry-run date. That produces 120 records and 72 evaluated questions.
A small, deterministic IDF-weighted lexical ranker orders the records against each question. It has no embeddings, model calls, long-term memory, or learned preferences. For each word budget, we take ranked records in order, without cutting a record in half, and count how many of the two required record IDs have been retrieved. The six budgets are zero, 35, 70, 140, 280, and 560 whitespace-separated words.
All fixtures are fabricated from a fixed seed. The answer key consists of two record IDs per question, not a guessed natural-language answer. Record ID matching makes the scoring reproducible, but also makes it substantially easier than real-world verification of whether a passage truly supports a claim.
Results
The table shows complete evidence coverage and the less demanding condition of finding at least one required record.
| Context budget | Both records | At least one | Questions |
|---|---|---|---|
| 0 words | 0 | 0 | 72 |
| 35 words | 0 | 72 | 72 |
| 70 words | 24 | 72 | 72 |
| 140 words | 72 | 72 | 72 |
| 280 words | 72 | 72 | 72 |
| 560 words | 72 | 72 | 72 |
A compact chart of complete cases makes the step visible. The scale ends at 72 questions.
0 words | 0 / 72
35 words | 0 / 72
70 words | ████████ 24 / 72
140 words | ████████████████████████ 72 / 72
280 words | ████████████████████████ 72 / 72
560 words | ████████████████████████ 72 / 72
Complete-evidence counts for 72 generated two-record questions, scored at six retrieval budgets.
At 35 words, every question has some support, but none has enough to meet the two-record answer key. This is the central caution: a search result that appears relevant is not necessarily sufficient. At 70 words, one third of questions include both records. The tested fixture reaches complete coverage at 140 words, and the next two budgets add no successes.
The plateau occurs at 140 words for this corpus, these questions and this ranker. The result gives a concrete budget threshold for the measured task. In a larger inbox, the ranker may put several irrelevant notes first; in a real task, facts may be hidden in attachments, revisions, or quoted history. More context can even make model reasoning worse if it introduces conflicting or stale information. We did not test any of those effects.
What it means for design
A context builder should ask whether the evidence for the task is complete, not simply whether a high-scoring record was retrieved. The distinction suggests tracking answer requirements separately: reviewer, date, decision, owner, document version, and similar slots. If the system retrieves the reviewer but no date, it should search again or present the gap rather than silently imply certainty.
It also suggests treating a context budget as a constraint to optimize against, not an invitation to fill the full window. The complete set may be small when record relationships are clear. The missing requirement may be better solved with a targeted follow-up retrieval than with another large slice of unrelated messages.
The private-world-model approach adds a separate problem: the right document may exist but not be authorized for the AI currently asking. Evidence coverage must therefore be measured after permission filtering, not by assuming every connected record can be passed to every model.
Measurement scope
This run deliberately avoids an AI model, so it cannot establish when an answer stops sounding generic. It cannot measure groundedness, citation accuracy, coherence, privacy, or user satisfaction. The lexical scoring, project identifiers, record lengths, and word budgets are exposed in the code; a different corpus could move the apparent threshold substantially.
The examples use tidy, short English records. They have no attachments, languages mixed in the same thread, deleted messages, timestamp disputes, or facts that contradict one another. The pass/fail criterion also assumes the correct evidence is identified ahead of time. Real personal search rarely has a perfect answer key.
Retrieving the missing second fact
A paired follow-on test uses longer, messier generated histories. One condition should use a fixed retrieved context; another should retrieve again when an answer requirement is still missing. Then ask a real model to answer under both conditions and independently grade its citations, omissions, and unsupported claims. Include the cost of additional retrieval, and apply identical permission boundaries to both arms.
For reproducibility, see experiments/context-budget/run.py and the companion raw.json in that directory. The generator's logic lives in experiments/run.py. The raw file records the ranking and number of gold records covered for each question and budget, so the table can be independently recomputed.
