Quantis tells you when its own answer can't be trusted.
A backtest can look convincing and still be wrong: a Sharpe ratio from a handful of bars, a result that only works with no trading costs, or an arithmetic bug nobody re-checked.
Below are ten integrity checks built into the result card. Each one fires when its assumption is broken, in plain English, next to the numbers it affects. One of them sits in a panel that is held back from the report for now, and its entry says so.
What the first check caught, in an audit run
Engine Sharpe about 5.8 · recomputed about 1.27 · off by about 78%
In May 2026 an audit run of a monthly-bar momentum strategy on three stocks tripped the integrity check: the Sharpe ratio the engine reported couldn't be reproduced from the equity curve. The gap is the difference between the two figures as a share of the engine's: (5.8 minus 1.27) divided by 5.8. The cause was monthly returns being annualised by √252 instead of √12. The fix was merged on 26 May 2026.
We'd rather show a gap than a wrong number.
Loud failures save real money.
The thresholds are written down.
Ten ways Quantis tells you the truth
Every mechanism on this page belongs to the result card the strategy agent returns. They run automatically, with no plan or setting needed. Where a panel is held back from the report for now, its entry says so.
Engine-recompute mismatch detection
The trap
A backtest engine can get its own arithmetic wrong. If nobody re-checks the headline numbers, a wrong Sharpe ratio looks exactly like a right one.
What Quantis does
After every backtest, the Sharpe ratio and CAGR are recomputed independently from the equity curve using the same conventions as the engine (risk-free rate, sample standard deviation, annualisation). If either differs by more than one part in a million (relative), the result card shows a mismatch warning naming the metric and the size of the gap. The "Arithmetic rechecked" badge only appears when the recompute agrees.
What you see in the product
In May 2026 an audit run of a monthly-bar momentum strategy on AAPL, MSFT and NVDA tripped this check: the engine reported a Sharpe of about 5.8, but the recompute from the equity curve gave about 1.27. The cause was the engine annualising monthly returns by √252 instead of √12. The fix was merged on 26 May 2026.
Insufficient-history suppression
The trap
A backtest with a handful of bars and one closed trade has almost no statistical power. A Sharpe ratio computed from 12 data points comes with a confidence interval so wide it tells you very little.
What Quantis does
Below 60 daily bars (about three months) or 6 monthly bars, the card does not show the headline metrics at all; it shows how many bars it had and how many it needs. Windows shorter than 12 months get a Short sample flag. With fewer than 10 closed trades, trade-level breakdowns carry a small-sample warning.
What you see in the product
A 7-month backtest carries a Short sample flag saying its metrics are provisional and may not generalise.
How far the Sharpe falls when the settings change
The trap
A strategy that looks great with a 20-day lookback but falls apart at 24 days is probably fitted to its parameters rather than capturing something real.
What Quantis does
We vary the strategy's key lookback parameters by ±10% and ±20% and re-run the backtest at each value. The card states what was varied, what was not, and the largest fall in the Sharpe ratio from the run as asked, as a share of that Sharpe: under 10% is a small change, 10% to 30% moderate, over 30% large. No label is given when the Sharpe as asked is under 0.20, because a share of a figure that close to zero means nothing.
What you see in the product
The settings line reads like: "Settings: large change (Sharpe fell by up to 44%, from 0.27 to 0.15)", with the range across the configurations that were run and the rule it was judged by.
Factor-data publication-lag disclosure
The trap
Fama-French factor returns are published monthly, a few weeks after each month ends. A backtest that runs up to today will always reach past the latest published month.
What Quantis does
When factor data doesn't cover the backtest window, the Factor Exposure panel says so and suggests an earlier end date. It doesn't estimate factor betas from missing data.
Not on the report at the moment: this panel is held back until its automatic check exists, so a result card does not show it today. The report's footer says how many panels are held back.
What you see in the product
Factor Exposure reads: "Data unavailable for this period. Fama-French factor returns are not yet published for the full backtest window", with a suggestion to re-run with an end date at least 6 weeks in the past.
Monthly-bar wide-error warning
The trap
Monthly data gives about 12 points a year, so any statistic computed from it is much noisier than the same statistic from daily data.
What Quantis does
We detect monthly data from the gaps between bars and flag it. The Sharpe standard error is computed from the actual number of monthly bars, and the integrity recompute annualises with 12 periods a year, not 252.
What you see in the product
A Monthly bars flag gives the bar count, notes that maximum drawdown misses moves within a month, and states the Sharpe standard error.
Time-slice stability, and held-out years for goal search
The trap
A strategy that worked in one stretch of history and not since may have been fitted to that stretch. The headline figure, which averages over everything, will mislead you.
What Quantis does
The card scores the same rules, unchanged, on successive slices of the test window. If the later slices' average Sharpe is 0.3 or more away from the periods before them, the card states the gap with its standard error and draws no conclusion beyond that. Nothing is re-fitted per slice and the slices overlap, so the card calls this a stability check and says it is not out-of-sample validation. Goal search is different: it chooses among its candidates on the first 70% of the window and reports how the chosen one did on the last 30%, which played no part in choosing it.
What you see in the product
A rule whose Sharpe was 1.1 in the periods before and 0.4 in the later slices gets a line under the stability panel: the later slices' Sharpe averaged 0.70 below the periods before them, with the standard error of that gap, and a reminder that the slices overlap and are not separate tests.
Could it be luck? Every test of the idea counts
The trap
Try enough variations of an idea and one of them will look good by chance. A Sharpe ratio on its own doesn't know how many tries it took to find it.
What Quantis does
Every backtest you run, every variant and parameter-sweep value computed beside it, and every goal-search candidate goes into your own test ledger. The verdict deflates the Sharpe ratio for the number of tests of the same idea (Bailey and López de Prado's deflated Sharpe) and says how many it counted and as of when. It also counts how many reshuffled versions of the strategy's own history had a big fall (a seeded block bootstrap of its daily returns, stored with the result: a picture of the history it had, not a forecast), and shows which market conditions the gains came from.
What you see in the product
The luck line reads like: N tests of this idea (runs and the configurations computed beside them), the Sharpe, the number of daily returns, and the deflated Sharpe ratio that results: how firmly the measured Sharpe stands above what the best of N tries with no skill would be expected to show. It is not a forecast.
Only companies that trade today are in the data
The trap
A backtest on today's list of companies leaves out every company that was delisted, acquired or went bust during the period. The survivors did better on average, so the backtest looks better than reality.
What Quantis does
Only shares and funds that trade today are in the data. A company that failed, was taken over or was delisted is not in it. So a test that picks from a list of shares is flattered, by an amount the test cannot measure. The result card says so in words on every test of shares: one wording when you named the shares yourself, another when the list is today's members of a group. A test on funds only carries no such note, because no company was picked. No figure is put on it, because none has been measured for these lists of shares.
What you see in the product
A test on shares you named carries this note: "Symbols chosen with hindsight: You named these symbols. A company that failed or was delisted during the test window could not be on a list drawn up today, so a test on hand-picked survivors is flattered by an amount this run cannot measure."
Total return, real costs and taxes
The trap
Backtests without trading costs, taxes or dividends misstate what you could have earned. Commissions, spreads, slippage, stamp duty and fixed broker fees all eat into returns, and dividends add to them.
What Quantis does
The headline result is total return: dividends are reinvested after withholding tax, and UK and Irish stamp duty is charged on purchases. Each order in a test pays 0.05% of its value in trading costs (4.5 basis points of slippage and 0.5 of commission), so a buy and a sell together cost 0.1%, or 10 basis points. The card itemises what costs and taxes took off, read from the engine's own report of what it charged. Give an amount and a UK broker and the run uses that broker's dealing charges in place of the commission, its FX fee and whole shares. Every trade is made at a day's closing price, one trading day after the closing prices that decided it, except a stop or a target, which fills during a day at its own price (or at that day's opening price when the price has already passed it). Each is an order at a price level, checked against every day's high and low after the day of the purchase. A stop fills at its level on a day whose low reaches it, or at that day's opening price when the day opens below the level. A target fills at its level on a day whose high reaches it, or at that day's opening price when the day opens above it. The level for a day is set from prices before that day, so a day's own high never lifts the stop it is checked against. A position bought at a day's close is not checked on that day. Daily prices do not show which came first inside a day, so on a day that reaches both a stop and a target the stop is taken. Each result also states its largest order as a share of the value of that share traded on an average day, and says so plainly when an order was above 1% of it.
What you see in the product
The Costs section lists what commission, slippage, spread, stamp duty, withholding tax and broker fees each took off the return, and the headline figures are after all of them.
An AI model has already read how the past turned out
A language model is trained on text up to a cutoff date, so anything it says about an earlier period can use knowledge nobody had at the time. We record which model wrote every AI answer, its training-data cutoff and when. When an AI-written research answer judges a period before that cutoff (“was X a good buy in 2022”), it carries a hindsight label: treat it as explanation, not as evidence a strategy would have worked. Answers built from our data (prices, fundamentals, macro series) are never labelled.
No strategy uses an AI-generated signal today; the agent says so if you ask for one. When they arrive, a backtest with an AI-derived input only reports headline metrics for the window after the model's cutoff. The earlier window is shown separately, marked contaminated, and if the clean window is under 60 daily bars there is no headline at all. The cutoff is a fuzzy line and a model's general knowledge still leaks, so this narrows the problem; it doesn't remove it.
Common ways backtests mislead
- A Sharpe ratio from a tiny sample, shown with the same confidence as one from decades of data.
- Testing on today's list of companies, which quietly leaves out the ones that failed.
- Results with no trading costs, presented as if they were achievable.
- Headline metrics nobody re-checks, so an arithmetic bug goes unnoticed.
- Numbers from an AI model's memory presented as if they were live data.
- An AI model's view of the past, written with hindsight, presented as if it could have been known at the time.
What Quantis does
- Below 60 daily bars (or 6 monthly bars) the headline metrics aren't shown; the card shows the shortfall instead.
- A plain note on every test of shares that only companies trading today are in the data. It does not fix the problem, and it puts no figure on it.
- Costs, taxes and dividends on every run, itemised from the engine's own report of what it charged.
- Sharpe and CAGR recomputed on every run, with a warning on any mismatch. That is how the √252 bug above was caught.
- In research answers and the summary on a result, a figure an AI model writes is checked against the data. A sentence with a figure that isn't in it is removed.
- AI answers name the model and its training cutoff, and judgements about earlier periods carry a hindsight label.
See it on a strategy you actually care about.
Describe a strategy to the agent and read the result card. Each check on this page either fires or stays silent, and the card shows you which.