Reliable Backtesting: The 5 Biases That Make Your Results Wrong

A reliable backtest is one that simulates real execution constraints: signals validated on the candle close, pessimistic order within the candle, transaction costs included, and a long, varied sample. Five biases are enough to flatter any method: repainting, survivorship, optimistic fill order, forgotten spread and a sample that is too short. Knowing them means being able to read a report in ten minutes.

Why a beautiful equity curve proves nothing

The equity curve is the most looked-at and least informative part of a backtest. It is the output of a chain of assumptions. If one of those assumptions is wrong, the curve stays pretty and becomes a lie.

A concrete example: a method that buys as soon as an indicator turns green during the candle, without waiting for the close, will show a win rate far above reality. Not because it is good, but because the history only keeps the cases where the green held. Live, you would have taken every false intrabar alert.

So the right reflex is the reverse of the habitual one: read how the test was run first, and only then the result.

Bias 1: repainting, the costliest lie

Repainting describes an indicator that alters its past signals. It comes in two forms.

• Intrabar repainting: the signal appears, disappears and comes back as long as the candle is unclosed. History only shows the final state.
• Retroactive repainting: the indicator uses future data to place a past point — typically those "perfect" top and bottom detectors.

The test is simple. Open your chart mid-candle, capture where the signals sit, then come back after the close. If anything has moved, every figure drawn from that history goes in the bin.

That is exactly why the PIPSTER PRO engine only validates a Heikin-Ashi Supertrend signal on the candle close: a displayed signal is a signal that already existed, at that moment, under the same conditions.

Bias 2: survivorship, what history no longer shows

Survivorship bias is famous in equities: testing a strategy on an index's current constituents automatically removes the companies that were delisted. Forex partly escapes it, but not entirely.

In retail trading it takes three sneakier forms:

• Instrument survivorship: you test on the pairs and cryptos that have trended nicely for three years, and ignore those that went nowhere.
• Settings survivorship: you keep the one parameter combination that produces a clean curve, out of fifty tried. That is curve-fitting in disguise.
• Period survivorship: you start the test "from 2023" because 2021 was less flattering.

Antidote: decide the period and the instrument list before running the test, and publish the instruments that performed badly too. A report showing only winners is a filtered report.

Bias 3: the optimistic order within the candle

This is the most technical and most common bias. On an H1 candle, if price touches both your stop and your first target, which happened first? OHLC data does not say.

Two possible choices, two different worlds:

With tight stops and 1R targets, the gap between the first two assumptions can be spectacular. A serious backtest states its choice explicitly. The backtest built into PIPSTER replays the series candle by candle with a pessimistic order: what you read is the unfavourable scenario, not the best case.

Bias 4: forgotten spread, commission and slippage

A "pure price" test assumes you buy and sell at the same price, free of charge. No trader lives in that world. Here is a full illustration on gold.

Hypothetical account of $10,000, risk of 1%, i.e. $100.

• Long entry on XAUUSD at 3,380.00
• Structural stop at 3,372.00, i.e. $8.00 of risk, so 1R = $8.00 of movement
• Size: $100 / (8.00 × $100 per lot per dollar) = 0.12 lot
• Target 1 at 1R: 3,388.00. Target 2 at 2R: 3,396.00. Target 3 at 3R: 3,404.00

Now add a round-trip spread of $0.30 and average slippage of $0.20 on entry. The real cost is $0.50 on $8.00 of risk, i.e. 6.25% of your R. Over 200 trades, that is 12.5 R evaporated in friction. A method with a gross expectancy of 0.08 R per trade drops to 0.017 R net: still mathematically positive, but practically at the mercy of the slightest spread widening around a news release.

The practical conclusion is not "give up", it is: the tighter your stop, the heavier the costs weigh. The position sizing and pip value calculators let you convert that friction into currency before you decide.

Bias 5: a sample too short or too homogeneous

Thirty trades over six weeks of clean uptrend test nothing: they describe an uptrend. A useful sample must be long and heterogeneous.

Three questions to ask of any report:

• How many trades? Below a hundred, the confidence interval is so wide that the average means almost nothing.
• Which market phases? You need at minimum an uptrend, a downtrend and an extended range.
• What is the longest run of consecutive losses? That number, not total profit, determines whether you will stick with the method.

On very low timeframes, one month of history produces plenty of trades but a single volatility season. On high timeframes, you get diversity but few occurrences. That trade-off must be made consciously, not by default.

The 7-step procedure for reading an honest backtest

1. Check for repainting. Test the indicator live against the history. If signals move, stop right there.
2. Read the close rule. Is the signal validated on the candle close, or while the candle is still forming?
3. Look for the intrabar order. Stop tested before target, or the reverse? If the report does not say, assume it is optimistic.
4. Add the costs. Average spread, commission, slippage. Recompute expectancy in R after friction.
5. Count the sample. Number of trades, period covered, market phases crossed.
6. Look at the worst run. How many losses in a row, and what cumulative loss in R? Multiply by your risk in currency: that is the number you will have to endure.
7. Replay it identically on another instrument or another period. Without touching a single parameter. If the structure of the result collapses, you were curve-fitted.

Allow thirty to sixty minutes for this procedure on a given method. That is little compared with the capital committed afterwards for months.

The legitimate objections, handled honestly

"What if the market gaps open on Sunday evening?" An honest backtest does not assume an exit at the exact stop price. If the open is beyond it, the loss is the open. On gold and indices these gaps do happen: your worst historical loss must be expressed in real R, not theoretical R. A trade exited at 1.7R of loss instead of 1R changes the risk-of-ruin maths.

"How long before I know whether it works live?" Far longer than the backtest suggests. If your method generates two signals a week, a hundred trades is close to a year. That is why a backtest serves to eliminate fragile methods rather than to certify one.

"Can a backtest replace the journal?" No. It measures rules, not your execution. The gap between the two is your real work.

A backtest does not predict the future. It tells you whether your rules would have been applicable, and at what psychological cost. That is already a great deal.

Putting these checks into practice today

You can apply this grid to any system, including the ones being sold to you on social media. Ask the seven questions: most spectacular curves do not survive the third.

If you want to test the approach on an engine that states its assumptions — validation on the close, pessimistic order, ATR or structural stop, targets in R multiples across 56 instruments and 22 timeframes — the PIPSTER PRO terminal is available in the Discovery pack at €0, and the seven calculators remain free for converting your R into currency. Start with one instrument, one timeframe, a hundred replayed trades: that teaches you more than ten curves found elsewhere.