Give me any ten years of price data and an optimizer, and I will hand you back a strategy with a 95% win rate and a triple-digit annual return. It will also lose money from the first week you run it live. This is curve fitting, and it is the single most common reason purchased EAs fail.
What curve fitting actually is
Optimization means adjusting strategy parameters until the historical results improve. Some of that is legitimate: choosing a sensible stop distance, filtering out low-liquidity hours. But past a certain point, the parameters stop describing market behavior and start memorizing historical noise. The strategy learns that buying EURUSD on the third Tuesday after a full moon worked in 2019. It did, once, by chance.
The warning signs
- Too many parameters. Every added parameter is a dial that can be tuned to the past. Our breakout system has a handful of rules you can explain in two sentences. Systems with 15+ optimized inputs are almost always memorizing noise.
- Perfect equity curves. Real edges have losing streaks. A backtest with no meaningful drawdown across a decade did not find an edge, it found the answer key.
- Fragile parameters. If a moving average of 47 produces great results but 45 and 50 lose money, the 47 is noise. Real edges are robust across nearby parameter values.
- No out-of-sample testing. Fit on 2015-2022, then test untouched on 2023-2025. If performance collapses out of sample, the strategy is fit, not found.
Trade count is your friend
Statistical significance needs sample size. A backtest with 150 trades can look brilliant by pure luck. Across 4,542 breakout trades and 1,438 mean reversion trades, luck averages out. Every extra trade in the sample makes it harder for randomness to masquerade as edge. When you evaluate any system, divide the years of history by the trade count. A ten-year backtest with 200 trades is 20 trades a year, which is a small sample wearing a long coat.
The live account is the final exam
The only test that cannot be curve-fit is the future. That is why we run our own systems on a live, MyFXBook-verified account and publish it, losing months included. A backtest is the hypothesis. Live trading is the experiment. Any seller who shows you only the hypothesis is asking you to fund the experiment.
The subtle version: survivorship in development
There is a quieter form of curve fitting that even honest developers fall into. You develop ten strategy ideas, backtest all ten, and publish the one that worked. No individual backtest was manipulated, yet the selection process itself was an optimization: with enough attempts, one strategy will always look good on historical data by chance alone. This is why our development process kills ideas out-of-sample: a concept gets fitted on the older data, then must independently survive on years it never saw, and then must survive live on real capital before it becomes a product. The Breakout and Mean Reversion systems are the two survivors of that funnel. The graveyard behind them is large, and the graveyard is precisely what makes the survivors credible.
Questions that expose curve-fit products in one email
You do not need the developer's code to test their honesty. Ask these, in writing:
- How many parameters does the system have, and what does each one do? (Honest answer: a short list with logic. Curve-fit answer: proprietary.)
- What happens to performance if each parameter moves 20% in either direction? (Honest answer: it degrades gracefully. Curve-fit answer: silence, because it collapses.)
- Which years were used for development and which were held out for validation? (Honest answer: specific years. Curve-fit answer: all data was used, sometimes phrased as we tested on the full history.)
- How many strategy candidates were tested and discarded before this one? (Honest answer: a real number. This question alone surprises developers who have never been asked.)
A seller who answers all four directly is rare and worth your attention. A seller who deflects has answered anyway.