Release note

Bootstrap confidence intervals on hit rate, profit factor, expectancy

1 min read
backtestaccuracytrust

A 0.55 hit rate sounds great until you realise it came from 24 trades, in which case the 95% confidence interval brackets coin-flip. We were quoting trade-level stats as point estimates without saying how confident we were — which is exactly the false-precision problem quant rigour is supposed to prevent.

Hit rate, profit factor, and expectancy are now reported with bootstrap 95% confidence intervals (1,000 resamples of the trade list). The interval shows up next to the point estimate as 0.55 [0.37, 0.73]. When the trade count is small, the interval is wide. When it's large, the interval narrows. Either way you're seeing the actual statistical quality.

If a strategy's "edge" disappears once you look at the lower CI bound, that's important information — and now it's visible by default rather than something you'd have to compute yourself.

Source sprint log: SPRINT_LOG-QUANT-GRADE-Q-PHASE2-B.md— transparency-by-default. The full log lives in the repo.