mrnh/rigor

Verified statistical inference for AI agents.
LLMs are decent at reciting statistics but bad at *doing* it reliably —
a t-statistic or a required sample size is a number recalled from
training data, not computed and checked. rigor is the alternative:
classical hypothesis testing (parametric and non-parametric),
correlation and regression, effect sizes, power/sample-size
calculation, multiple-comparisons correction, and always-valid
sequential testing for results checked more than once before they're
final, computed from scratch and returned as a cited, assumption-
checked answer -- plus a decision helper for picking the right tool and
a batch tool for running/correcting many comparisons at once, since
"which test do I even use," "I forgot to correct for multiple
comparisons," and "I peeked at the dashboard and stopped early" are
their own common failure modes, distinct from getting a single formula
wrong.
**A concrete case where this matters.** The one sample-size number
everyone half-remembers is Cohen (1988)'s own worked example: d=0.5,
alpha=.05, power=.80 -> n≈64 per group. It's in every textbook and
slide deck, so it's also what gets pattern-matched to when a
*similar*-looking question comes up. Ask instead for d=0.46, power=.85
-- a modest, realistic revision, not a trick:
85, not "about 64" -- a third more participants to recruit than the
half-remembered number suggests, from a question that *looks* like the
famous one. The formula itself isn't hard (power.py runs the same
bisection search either direction, in a few lines); the failure mode
is that recalling a nearby-looking answer feels indistinguishable from
computing the right one, right up until the number's wrong.
**A second concrete case.** An agent (or person) watching a live
experiment's dashboard and checking the p-value every time new data
comes in, stopping the moment it clears 0.05, is a textbook way to
fool yourself -- and it's the default way anyone actually monitors a
running experiment, fixed-sample design or not:
Checking 10 times at a nominal 5% level is really running at closer to
19% -- roughly one in five "significant" results would be noise even
with zero real effect. rigor sequential proportion / rigor
sequential mean
compute an *always-valid* p-value instead (mSPRT, Johari et al. 2017):
checkable after every new observation with the false-positive rate
actually staying at the nominal level, no pre-committed sample size and
no correction for "how many times have I looked" required.
Built as an MCP server: a scan of the current MCP ecosystem (Context7
for coding docs, several physics/engineering/chemistry/geo servers,
even Bentley's STAAD integration) found statistics/experimental design
as one of the few common agent needs nobody had covered yet.
The statistics themselves (rigor/distributions.py, inference.py,
nonparametric.py, correlation.py, regression.py,
effect_size.py, power.py, corrections.py, plus the decision/batch
helpers in advisor.py and batch.py) are pure standard library, no
dependencies. The package as a whole does depend on the official mcp
SDK, since the MCP server is a first-class part of what it ships, not
an add-on -- see Install.
Install
(the PyPI distribution is rigor-mcp since plain rigor was already
taken by an unrelated package; the importable package and the CLI
command are both still just rigor.) This gets you both console
commands, rigor (CLI) and rigor-mcp (MCP server) -- deliberately
one install, no extras to get right, since uvx rigor-mcp (how most
MCP clients would actually invoke this) has no way to request an
extra.
What's in it
- **rigor/distributions.py** — t, chi-squared, and F distributions
built from scratch on stdlib (regularized incomplete gamma/beta),
verified against exact closed-form identities (t(1) = Cauchy,
chi2(2) = scaled exponential, t² = F(1, df)) rather than trusted