Every market opinion arrives with evidence attached. The columnist who warned about the euro, the analyst who called the oil spike, the strategist who saw the dollar turn: each can point to a chart that moved the way they said it would. What almost never comes attached is the record of everything else the same person predicted that year.

Traders have a cheap way to confront this problem, and it is worth borrowing even if you never place a trade. Tools such as FX Backtest replay historical price data bar by bar, hiding what comes next, so a rule has to be applied before the outcome is visible rather than explained after it.

A clean story is the easiest thing to build

Macro commentary runs on narrative. When French debt dominates the headlines, a reader can find a well-argued case that Europe is heading for another eurozone debt crisis, and the euro's recent weakness will sit neatly inside it. When the Strait of Hormuz closes, every oil-linked currency move becomes obvious in retrospect.

None of that is dishonest. The trouble is that a story told after the move costs nothing to tell. Replay strips the comfort away, because you are sitting on a candle in the middle of a crisis with no idea whether the next week is the bottom or the start of a longer slide, which is the position everyone was actually in at the time.

A backtest is mostly a record of discipline

The useful part of testing is not the equity curve at the end. It is that you had to write the rule down first: what the setup is, how much you risk, where the stop goes, when you stay out. Vague conviction cannot be tested. Only a specific claim can fail.

Sample size matters in the same way. A rule that survives twenty trades in one quiet year has proved nothing about trending, ranging and news-driven markets. A hundred or more trades across different conditions starts to say something, especially if the sample includes a drawdown that was genuinely uncomfortable to sit through.

Costs belong in the test too. Spreads widen around rollover and around data releases, commission is charged per lot, and real fills slip. A strategy that earns its edge in fractions of a pip usually loses it to these frictions, and finding that out on historical data is far cheaper than finding it out live.

The number nobody reports

There is a sharper trap, and it has been documented. In the Notices of the American Mathematical Society, David Bailey, Jonathan Borwein, Marcos López de Prado and Qiji Jim Zhu showed that impressive simulated performance is easy to produce after testing only a modest number of strategy variants. The more variants you try, the more likely the winner is noise dressed as skill. Their key point is about disclosure: analysts rarely say how many configurations they ran, so nobody else can judge how overfitted the result is.

That applies well beyond quantitative finance. A commentator who has published forty predictions and reminds you of the two that landed is running the same selection process on their own output, usually without meaning to.

Reading, and testing, with that in mind

The practical upshot is a short set of questions. How many versions of this idea were tried before the one being shown to you? What result would have counted as the idea failing? Was the call recorded before the outcome, or assembled afterwards from the parts that fit?

Those questions are easier to ask of a trading rule than of an opinion column, which is exactly why the exercise is instructive. Run a strategy over a few hundred historical bars, note how much of your original confidence survives, and the habit transfers to how you read everything else.

It is worth stating the limit plainly. A backtest is a simulation, prepared with hindsight and with no money at stake, so it cannot promise anything about the future. Trading foreign exchange involves a substantial risk of loss, and testing a strategy reduces your ignorance rather than your exposure.

© qolumnist