Monte Carlo & parameter stability

Your backtest is one sample from a distribution of possible outcomes. Resample it to see the whole distribution, and map the parameter space to check your edge is a plateau, not a lucky spike.

A backtest gives you a single number — "34% return, 12% max drawdown" — and single numbers are seductive and dangerous. The exact order in which your trades happened was partly luck. Reshuffle that luck and you would have gotten a different drawdown, a different final equity. The single backtest is one draw from a distribution of outcomes you never see. Monte Carlo methods let you see the whole distribution.

Trade-shuffling and resampling

The simplest, most useful Monte Carlo for trading: take your list of individual trade returns, and randomly resample them thousands of times to build thousands of alternative equity curves. Each one is a plausible universe in which the same strategy, with the same per-trade edge, happened to deal its trades in a different order (or drew a slightly different sample of them). From those thousands of universes you read off a *distribution* of final returns and, more importantly, of drawdowns.

import numpy as np

class="c"># Per-trade returns from your backtest, e.g. [+class="n">0.012, -class="n">0.008, +class="n">0.021, ...]
trades = np.array(backtest_trade_returns())
n = len(trades)

final_returns = []
max_drawdowns = []

rng = np.random.default_rng(class="n">0)
for _ in range(10_000):
    class="c"># Resample the SAME trades in a random order (with replacement).
    sample = rng.choice(trades, size=n, replace=True)
    equity = np.cumprod(class="n">1 + sample)

    final_returns.append(equity[-class="n">1] - class="n">1)

    peak = np.maximum.accumulate(equity)
    dd   = (equity - peak) / peak
    max_drawdowns.append(dd.min())            class="c"># most negative = worst drawdown

final_returns = np.array(final_returns)
max_drawdowns = np.array(max_drawdowns)

print(fclass="s">"Median return:        {np.median(final_returns):class="n">6.1%}")
print(fclass="s">"5th percentile return:{np.percentile(final_returns, class="n">5):class="n">6.1%}")
print(fclass="s">"Median max drawdown:  {np.median(max_drawdowns):class="n">6.1%}")
print(f"95th pct worst DD:    {np.percentile(max_drawdowns, class="n">5):class="n">6.1%}  <- plan for THIS")

This reframes everything. Your original backtest showed a 12% drawdown? The Monte Carlo might reveal that in 1 out of 20 equally-plausible orderings, the drawdown was actually 28%. That 28% is not a freak — it is a normal member of your strategy's family, and it is the number you must be financially and emotionally able to survive. You do not get to choose which universe you live in. Size your risk for the bad tail, not the pretty median.

Parameter stability: plateau, not spike

The second robustness test attacks overfitting from a different angle. Take the parameter you settled on and ask: *what happens to performance if I nudge it?* Plot performance across the whole neighbourhood of parameter values. There are only two shapes it can take, and they tell you everything.

The blue strategy sits on a plateau: the chosen lookback of ~7 is good, but so is 6, and 8, and 9. If the future shifts the optimal value slightly — as it always does — you barely notice. The violet strategy sits on a spike: one magic value scores brilliantly and every neighbour collapses. That is not an edge; that is a coincidence you tuned to. In live trading the peak will drift off your magic number and you will get the failing neighbours instead.

For strategies with two parameters, generalise this into a heat map: a grid of performance across both parameters at once. You want to see a broad, warm region of good results — a robust zone — not a single hot pixel surrounded by cold. A lone bright pixel is the visual signature of overfitting; a warm continent is the signature of a real edge.

Putting it together: the robustness mindset

1. Assume your backtest is optimistic. It is one lucky-or-unlucky draw, fit at least a little to the past. Start from suspicion, not excitement.
2. Split time honestly. Train/test, then walk-forward — and treat your out-of-sample look as sacred and single-use.
3. Hunt your biases. Lag every signal; use point-in-time data; re-derive one trade by hand to check for look-ahead and survivorship leaks.
4. See the distribution. Monte Carlo your trades and plan capital and nerves around the bad tail, not the median.
5. Demand a plateau. Prefer robust, forgiving parameters over fragile optimal ones. Ugly-but-stable beats beautiful-but-brittle every single time.

The market can produce any pattern you can dream up, given enough searching. Robustness testing is the discipline of assuming you fooled yourself — until you have proven, out of sample, that you did not.

You now have the honesty toolkit: you can name why backtests lie, catch the biases that manufacture fake profit, and stress-test an edge until only the real part survives. With Part 7 we turn to keeping the real edge alive — risk management and position sizing, the machinery that decides whether a genuine edge compounds or gets wiped out by a single bad tail.