What a backtest can — and can't — tell you
Backtests are invaluable for rejecting weak ideas and understanding behaviour, but easy to misread. Here's how overfitting, bias and costs mislead — and how to read results responsibly.
Quant HFT AI · · 3 min read
A backtest replays a trading strategy against historical data to estimate how it might have behaved. It is one of the most useful tools in systematic trading research. It is also one of the easiest to misread. This guide explains what a backtest can genuinely tell you, where it misleads, and how to read results responsibly.
What a backtest is good for
Used carefully, a backtest can:
- Find errors in logic. If the rules don’t behave as intended across thousands of historical bars, you have found a bug cheaply.
- Reject weak ideas early. If a hypothesis fails even under generous assumptions, it is unlikely to work in live markets.
- Characterise behaviour. How often does the strategy trade? How long are losing streaks? How does it behave in high-volatility periods?
- Compare variants consistently. Run every variant under identical assumptions so the differences mean something.
In other words, a backtest is best at falsifying ideas and describing behaviour, not at forecasting profits.
Where backtests mislead
Overfitting
Every adjustable parameter gives a strategy more freedom to fit historical noise. A system tuned until its equity curve looks smooth has often learned the past rather than a durable market effect. Warning signs include many parameters, very specific values (a 37-period moving average outperforming 35 and 40) and performance that collapses when settings change slightly.
Look-ahead bias
Look-ahead bias occurs when a backtest uses information that would not have been available at the time of the decision. Common causes include using a bar’s closing price to decide trades within the same bar, or using revised economic data instead of the values first published.
Survivorship bias
Testing only on instruments that still exist today ignores those that were delisted, merged or failed. This flatters results, particularly for equity strategies.
Unrealistic costs and fills
Spreads widen, slippage happens and orders are not always filled at the price you wanted, especially around news and at market open. A strategy with many small trades can turn from profitable to loss-making once realistic costs are included.
Regime change
Markets change. Volatility, liquidity, market structure and participant behaviour shift over time. A strategy that thrived in one regime may struggle in the next.
How to read results responsibly
- Separate research data from validation data. Develop on one period; evaluate once on data the model has never seen (out-of-sample).
- Use walk-forward testing. Repeatedly optimise on a rolling window and test on the following period. This shows how the process, not just one parameter set, holds up.
- Stress the assumptions. Double the costs and add slippage. Remove the best trades. If the result depends on a handful of outliers, be cautious.
- Check parameter stability. Good regions should be broad plateaus, not narrow peaks.
- Forward test before risking capital. Paper or demo trading reveals execution differences no backtest can model.
- Write down the assumptions. Data source, period, costs, slippage model and known limitations should travel with every result.
The honest conclusion
A backtest can tell you that an idea is not obviously broken and give you a realistic picture of its behaviour. It cannot tell you what will happen next. Treat every result as evidence to be challenged, not as a promise.
This article is educational and is not financial advice. Trading involves substantial risk of loss, and past or simulated performance does not guarantee future results. See our Trading Risk Disclosure.