Numbers
How well Caute finds what an assistant needs, measured on a published set, so the claim has receipts. Every figure here is retrieval only: no model answered anything, so the number is Caute's, not an assistant's.
Retrieval recall at N tokens
Measured on 2026-10-19 over the agent context set: twenty tasks phrased the way an assistant asks them of a library ("draft the background section on how the kelp barren is changing and why"), each with the passages a good answer needs, over a corpus of forty-five documents (pages, PDFs, email with attachments, notes) and four highlights. A passage counts when what came back overlaps it. The held-out half is the one that counts.
| Tool | Gold passages found | Tokens returned |
|---|---|---|
search, its excerpts alone |
0.625 | 60 |
search then get_item on each result, a page each |
0.625 | 466 |
gather at its default budget of 2,000 tokens |
0.812 | 389 |
gather is one call: the best passages for a task, grouped by card, near-duplicates removed, the card's
confirmed properties before its passages, and the person's own highlights first. The excerpts search returns
are a line per card, which is why an assistant reads the cards after it; the second row is what that costs.
Search
The same corpus carries eighty-seven queries of every kind people type (identifiers, keywords, paraphrases, questions nothing saved answers), graded by hand. On the held-out half, nDCG@10 is 0.884, recall@10 0.792, and search says nothing to 0.917 of the questions nothing saved answers.
Long conversation histories: LongMemEval
LongMemEval is a public benchmark of long chat histories (Wu et al., ICLR 2025), used here in its cleaned release, LongMemEval_S, under its MIT licence. Each question comes with its own history of about 48 past conversations, roughly 122,000 tokens, and the benchmark marks which conversations, and which turns, hold the answer. Measured on 2026-10-21: 60 questions, ten drawn from each of its six types (seed 18), every conversation saved as a conversation card the way a chat export is, each question searching only its own history.
Each row is what an assistant would be handed. Session recall (any) is how often that held at least one conversation with the answer, (all) how often it held every one, and turn recall the share of the answer's turns it held.
| What the assistant is given | Session recall (any) | Session recall (all) | Turn recall | Tokens |
|---|---|---|---|---|
| The whole history | 1.000 | 1.000 | 1.000 | 122,455 |
gather at 2,000 tokens |
0.983 | 0.867 | 0.567 | 1,471 |
gather at 8,000 tokens |
1.000 | 0.883 | 0.598 | 2,457 |
gather at 2,000 tokens, keyword search only |
0.783 | 0.583 | 0.330 | 813 |
| The most recent conversations, 2,000 tokens of them | 0.067 | 0.050 | 0.031 | 1,708 |
| The most recent conversations, 8,000 tokens of them | 0.167 | 0.067 | 0.093 | 7,750 |
Handing over the whole history finds everything, at about 122,000 tokens a question. Keeping only the most
recent conversations, the simplest way to remember, rarely holds the answer. gather finds an answer
conversation for 59 of the 60 questions in about 1,500 tokens; it is weakest on questions about the person's
preferences and on questions that need several conversations or a date to line up, where it often has
one of the conversations and not all of them. At 8,000 tokens it uses about 2,500: past that, more budget
buys little.
In the form the benchmark reports itself, a conversation with the answer is among search's top 5 results
for 0.983 of questions, and the same among its top 10. These are retrieval numbers only, so they are not
comparable with scores that grade an assistant's answers.
What changed and what did not
A boost for cards the person highlighted was tried: it lowered the held-out nDCG@10 from 0.884 to 0.879, so it ships off. A marked passage is still shown first on its card and says "your highlight".