Evidence
Research & benchmarks
Every performance claim on this site traces back to a committed benchmark artifact in our research repository — currently private, with the papers pending arXiv acceptance. The harness is available on request. No numbers are quoted here that you cannot reproduce from a tagged release.
Suites
- LongMemEval
- RULER 4–32k
- E7 synthetic
- Throughput harness
What the evaluations show
Positronic is evaluated on established memory benchmarks rather than self-designed demos. Each run commits its metrics JSON so the figure can be re-derived, and the paper cites the engine tag that produced it.
LongMemEval · n=50
Recall 0.90 vs 0.12 without memory
The archival and local profiles reach 0.90 against 0.12 for the no-memory baseline — a Δ of 0.78, with a fallback rate of 0.0 and perfect recall on the committed metrics-50 run.
LongMemEval · n=500
Recall 1.000 across all six question types
Scaling the same protocol to 500 trials holds recall at 1.000 with accuracy 0.262 versus 0.202 without memory. Retrieval never failed; the weakest category was temporal reasoning at 0.180, which is an extraction ceiling rather than a retrieval miss.
RULER · 4k–32k
Flat 242 tokens against a growing haystack
Top-8 retrieval uses a constant ~242 tokens while the full-context haystack grows from roughly 2,249 tokens at 4k to 18,000 at 32k — between one tenth and one seventy-fourth of the context, with recall@1 preserved at 1.0.
E7 synthetic
Retention curves behave as designed
Fifty-five events tracked over seventy-eight weeks separate exactly as the profiles predict: archival and long_term retain all fifty-five, balanced thirty-five, short-term eleven. Short horizons make every profile look alike; long horizons is where the curves split.
Throughput
About 6,000 events per second
Synthetic batch ingestion reaches roughly 6k events/s. On a realistic mix the salience gate encodes about five percent of incoming events, which is what keeps storage and recall cheap over time.
Store layer
p95 of 0.7 milliseconds
SQLite with FTS5 and reciprocal-rank fusion returns stored episodes in under a millisecond at the ninety-fifth percentile — the sub-millisecond figure quoted for the store, distinct from the end-to-end pipeline.
Method
Two numbers, never one
Memory systems are usually quoted with a single latency figure that quietly omits the model calls. Positronic reports the store and the pipeline separately, because they are genuinely different things: the store is instant, while the pipeline takes roughly what the language models take.
On real LongMemEval runs the end-to-end pipeline lands around seventy-three seconds per question at n=500 — ingest, embed, two model passes and a judge — while the retrieval sub-component alone is about 135 milliseconds. Reporting one without the other would be misleading, so we report both.
Reproduce it
Engine tests run with pytest engine/tests/ from the repository root. Committed artifacts include figs/metrics-50.json, figs/metrics-500.json and the E7 survival figure, each tied to a tagged release such as v0.3.0.
PAI pins the engine tag it was validated against, so a benchmark figure and the code that produced it stay linked.
Thesis
Curation, not storage, is the hard problem
Human memory is not one index — it is a federation of small specialised subsystems, none intelligent alone. Positronic takes that seriously: a salience gate decides at encode time what deserves to survive, decay runs on subjective time instead of wall-clock, and entities are consolidated into objects that persist across episodes. The research repository documents the architecture behind each of those choices, along with the reasoning for why demotion beats deletion.