Five Backtesting Mistakes That Quietly Inflate Results
Lookahead bias, survivorship, overfitting, ignored costs, and in-sample reporting — the subtle errors that make a backtest look better than reality.
A backtest is a hypothesis about the past, not a promise about the future. Even honest researchers produce misleading curves when small methodological errors creep in. Here are five of the most common — and how to catch them.
Educational methodology only. None of this is advice or a description of a tradeable strategy.
1. Lookahead bias
Using information that wasn't available at decision time is the classic killer. The fix is a strict rule: decide on closed bars, execute on the next bar.
position = signal.shift(1).fillna(0) # signal known at close[t] acts from t+1We cover this in depth in Avoiding Lookahead Bias in Backtests.
2. Survivorship bias
Testing only instruments that still exist today excludes everything that failed. Your universe should reflect what was actually available at the time, delisted names included.
3. Overfitting to the sample
Tune enough parameters and any dataset will yield a beautiful curve. Reserve out-of-sample data, prefer fewer parameters, and treat a result that only works in one narrow configuration as noise.
4. Ignoring costs
Fees and slippage widen the bar a strategy must clear. A frequently-trading system can look great gross and be negative net. Estimate the drag with the Brokerage Calculator.
5. Reporting in-sample numbers
The number you optimized on is not evidence. Report results on data the model never saw, and pair returns with risk — a high CAGR next to a deep max drawdown tells a very different story than the return alone.
The mindset
Assume your backtest is wrong until you've ruled these out. Building the decision/execution split and trailing-window features into your engine from day one makes the honest version the default.