When a provider shows performance numbers for an AI trading signal, those numbers were generated one of two fundamentally different ways — and the difference matters more than most marketing pages let on.
Backtesting: simulating the past
A backtest runs a strategy's rules against historical price data to see how it would have performed, after the fact. It's a genuinely useful research tool — it's how quantitative researchers first screen an idea before risking real capital on it. The problem isn't backtesting itself; it's how easy it is to produce an excellent-looking backtest that doesn't reflect anything durable.
Why backtests are easy to make look good
- Overfitting — adjusting a strategy's parameters repeatedly until it fits the historical data extremely well, which often just means it has learned the noise in that specific dataset rather than a real, repeatable pattern.
- Look-ahead bias — accidentally using information in the simulation that wouldn't have been available at the time (e.g., a data revision that only appeared later).
- Survivorship and selection bias — testing only on instruments or time periods that happened to work well, and quietly excluding the ones that didn't.
- Ignoring real-world costs — spreads, commissions, and slippage are easy to model too generously or leave out entirely, which flatters results.
Forward testing: performing on data the model hasn't seen
Forward testing (also called out-of-sample or live/paper testing) means running the strategy on new data as it arrives, after the strategy's rules were already fixed — so there's no way to have tuned the rules to fit results that hadn't happened yet. A forward-tested track record, especially one published continuously and timestamped on an independent platform, is much stronger evidence than a backtest alone.
What to actually ask a provider
- "Is this backtested, forward-tested, or live account performance?" — a credible provider will answer this directly and specifically.
- "Over what period, and does it include the recent losing periods, not just the highlights?"
- "Are trading costs (spread, commission, slippage) included in these numbers?"
A provider that can't or won't answer these clearly is giving you less information than the length of their results page might suggest. This ties directly into the broader checklist in our guide on evaluating a signal provider, and connects to what AI trading signals actually are under the hood.
The honest takeaway
Neither backtesting nor forward testing can prove a strategy will keep working — markets change. But they're not equally informative: a backtest alone tells you a strategy could be curve-fit to history, while a long, consistently published forward-test record at least demonstrates the rules were fixed before the results were known. Treat the difference as a filter, not a guarantee.