The verdict now shows big falls in its reshuffled history, where the gains came from, and whether it could be luck

Update, 4 October 2026: the report no longer words these figures as a chance. The first line is now titled "Big falls when its history is reshuffled": it is the share of 1,000 rearrangements of the run's own daily returns that fell more than 20%, not the probability of anything. The luck line gives the deflated Sharpe ratio, which says how firmly the measured Sharpe stands above what the best of the tries would show with no skill. The wording below is as it was on the day.

Three pieces of evidence that were buried in the statistics, or didn't exist, now sit in the verdict at the top of every backtest result. Each one opens an explanation with every input, so you can check the figure yourself. On 28 September we added two more checks: whether every condition in a combined rule mattered, and a number-by-number check of the AI-written summary.

Chance of a big fall? We resample the result's own daily returns in blocks of about ten trading days (a stationary block bootstrap) to make 1,000 alternative versions of the same history, each as long as your backtest. The line counts how many fell more than 20% at some point and how many ended below where they started. The random numbers come from a fixed seed and the figure is stored with the result, so it is the same in chat, on your saved strategy and on its share page, and doesn't change when you reload. When we built it, a five-year golden cross on SPY showed 311 of the 1,000 versions falling more than 20% (31%) and 29 ending below the start (3%). Since 28 September the line also states the horizon, for example "of this 5-year history". It isn't shown for runs with less than about a year of daily data, and the explanation says why.

Where the gains came from. The same line says which kind of market the gains came from, judged by the benchmark's trend (up, down or no clear trend) and whether it was calm or volatile, and names the kind of market where the strategy lost most. It says "most of its gains" only when one kind of market really holds at least half of them, and otherwise says the gains were spread out. Only kinds of market with at least 60 trading days in the test are named. The labels describe the benchmark's recent trend rather than each day's move, and the explanation says so.

Could it be luck? Every test you run is recorded in a private test ledger. Tests of the same idea (the same strategy type on the same symbols in the same currency) are counted together, along with the variants run inside each test. The verdict then gives the chance that the strategy's true Sharpe ratio is above zero, allowing for how many versions of the idea you tried. In our worked example, a first test of a golden cross on SPY, with three versions, read 93%; the same user's seventh test of the idea, 21 versions in all, read 83%. If the ledger can't be read, the line says "Not available" rather than showing a number. The list of your tests is visible only to you. Share pages show the count and the result, and since 28 September no longer show the spread of your other tests' Sharpe ratios. For quants: this is the deflated Sharpe ratio of Bailey and López de Prado, with N the number of distinct tests and the expected maximum Sharpe taken from the spread of their Sharpe ratios; the probabilistic Sharpe ratio underneath uses daily log returns without the risk-free rate, as stated on the card.

Did every condition matter? (from 28 September) For rules that combine conditions, the engine now works out every buy and sell decision again with each condition removed, and counts the decisions that come out differently. If a condition never changed anything, the verdict says so. On five years of SPY, "golden cross but only buy when RSI is under 70" gets "Did every condition matter? No": the RSI filter never blocked an entry, so the result is the same as the golden cross alone.

The AI summary is checked number by number (from 28 September). The written summary under the verdict comes from an AI model. Every number in it must now match a figure from your result as displayed, and every "beat" or "trailed" must agree with the strategy's and the benchmark's returns. A sentence that fails is removed and listed with the reason. If more than half fail, the AI summary is withdrawn and a plain template is shown in its place, labelled as such.