Research

---

We register our studies before we run them, and we publish the result either way. Every study below links to the registration filed before the data existed, the preprint, and the code behind the analysis.

We set what we registered before the data existed beside what the data showed. When a result went against our own hypothesis, we made it the headline, because how an organization reports results that go against it is the most useful thing you can know about how it reports everything else.
 

We predicted a published null result would reproduce. It held in two of eight settings.

Our registered reproduction of a 2026 result on chain-of-thought reliability tested two claims: that the shape of a model's entropy trajectory predicts a correct answer, and that the size of its entropy drop does not. The shape signal replicated at full test-set scale. The null held in two of the eight model-and-benchmark settings we ran. On our anchor model the rank correlation was -0.018 on GSM8K and +0.414 on MATH-500. We report that division by setting as the finding.

· Preprint: arXiv:2609.19606

· Registration: osf.io/8w2q3

· Code: github.com/ai4altruism/entropy-drift-pilot

We predicted a compiled wiki would cost less per query than retrieval. It cost 21× more.

Our registered hypothesis was that compiling a research corpus into a maintained wiki would pay for itself across many queries. Measured, the wiki spent about 21× more per query than vector retrieval. The amortization we had predicted is not merely unsupported but unavailable: the break-even calculation returns a negative number of queries. We made that reversal the paper's headline rather than a limitation.

· Preprint: arXiv:2605.18490 

· Registration: osf.io/zemhp 

· Code: github.com/ai4altruism/rag2compare
 

We set a reliability threshold in advance, and our own grading did not clear it.
 

Our progressive-disclosure ablation registered a Cohen's κ of 0.60 as the required agreement between human and automated grading. The pooled result was 0.23. It appears in the paper's abstract rather than its appendix, together with the condition under which our non-inferiority claim does not hold.

· Preprint: arXiv:2607.04576 

· Registration: osf.io/feka7 

· Materials: osf.io/a53mz
 

We tested whether context prompts make transcription more accurate. They did not.
 

A preregistered ablation on archival oral-history audio found no detectable change in aggregate word error rate from prompt-level context, a null result against our own working assumption. The effect that does exist is narrower and invisible to the headline metric: recall of specific listed terms improves while aggregate error stays flat. That is a finding about the measure as much as about the method, and it changed what we build.
· Preprint: arXiv:2608.28875

· Registration: osf.io/ns49b 

· Materials: osf.io/9nsvr
 

AI systems are increasingly evaluated by scores whose validity is rarely argued explicitly. Our research is about that gap: whether a measurement supports the claim being made on it, and what it takes to know. An organization making that argument has to be able to survive the same test, so we register the analysis before we look at the data, publish results that go against us, and ship the artifacts that let someone else disagree with us on the evidence.

The analysis code for every study above is public. The reproduction kit for the retrieval comparison is at github.com/ai4altruism/rag2compare, and the harness and analysis scripts for the entropy reproduction are at github.com/ai4altruism/entropy-drift-pilot. The code for the other two studies is deposited on OSF with their materials. Other tools and infrastructure are at github.com/ai4altruism.

Patent application. System and Method for Progressive Learning Using Iterative Assessment and Generation. U.S. Patent Application #18/964,674, filed March 2025.