Validating a Strategy Without Fooling Yourself

A good-looking backtest proves very little. It only shows that there is a way of looking at the past in which the strategy makes money – and you chose that way after seeing the data.

This guide shows you how to build tests a strategy can actually fail. If it passes them, you have a serious clue. If it fails, you have saved money and months of work.

What makes it different

• Real data, not invented numbers: every example uses 12 years of EURUSD H1 data (2015–2026) exported from MetaTrader 5, plus a test on 21 currency pairs.
• One example strategy followed through every chapter, so you see each tool at work, step by step.
• Real failures as well as a real success: ideas that looked great and collapsed out of sample, and a real indicator (VolBand) that passed a pre-registered test.
• Three Python scripts included that reproduce every number in the guide on your own data.

What you will learn

• Why your backtest fools you: too many variants, too few trades, a simulation that is not the market
• Pre-registration: writing the rules before you look at the data
• In-sample, out-of-sample and holdout, and how to use them without cheating
• Walk-forward done right
• Parameter sweeps: looking for plateaus, not peaks
• From the simulator to MT5: costs, spreads and the gap you must measure
• How many trades you really need: t estimate, bootstrap and permutation tests
• Validating a filter, not just a signal, with a random-filter control
• Forward validation before going live
• A diary of failures, and a final checklist to use before trading, publishing or buying a strategy

Who it is for

Traders who build, buy or sell automated strategies, indicators or EAs and want to know whether the results are real. You need basic trading knowledge and elementary statistics; coding helps but is not required.

What you get

• The guide in PDF, 46 pages, with charts, worked examples and “In practice” boxes
• A ZIP with three Python scripts (NumPy and pandas) and step-by-step instructions

What you won’t find

Ready-made strategies that “work”, promises of returns, or signals. You will find a method for judging any strategy, including your own.

About the author

Dario Cadeddu develops quantitative indicators and tools for MetaTrader 5, validated with pre-registered tests and published on mql5.com.

Disclaimer

For educational purposes only. This is not financial advice. Past results, even when carefully validated, do not guarantee future results.