Every test now checks itself against luck

A rule can come out ahead of simply holding by luck. From now on every strategy test checks for that, at no extra cost.

The luck check. The same rule is run 50 times buying and selling on random days, with the same time in the market, and 50 times on the same prices put in a shuffled order. It is a lighter version of two of Prove it's attacks, with the same pass marks. The result now has a line for it: "Luck check: did it beat chance?". Most checks finish with the result; a slower one shows "running" and fills in by itself.

What it changes. A rule that ended above simply holding is shown as a plain "Yes" only when the luck check passed. Otherwise the answer says why not: "Yes, but this could be luck", or that the check is still running or could not run. The short card for new accounts says "Ended above simply holding" in the same cases, instead of "Beat".

Could it be luck? This line printed a score such as "82%", which was easy to read as "82% luck". It now answers the question first ("Yes, it could be" under 95, or "It clears the 95% bar"), then gives the score and what it is not. It also counts every try of the same idea on your account, and the deflated Sharpe in the full statistics now counts the same tries.

Many tests. The warning about running many backtests now says it counts every idea you tested in the last 30 days, and how that differs from the count behind "Could it be luck?".

Reshuffled history. The paths behind "Big falls when its history is reshuffled?" are now 10,000 instead of 1,000, in blocks whose length is worked out from each test's own returns. Under them, a new panel shows how the rule did through the market's larger falls that lie inside the test. Results made before keep their own figures.

These describe the past. None of them is a forecast.