Overfitting: the cardinal sin

Why more parameters and more tweaking always produce a prettier backtest and a worse live result — and how to feel it happen in code.

There is exactly one way to make a backtest look amazing: keep tweaking it until it does. This is not a joke — it is the single most dangerous fact in algorithmic trading, and almost every blown-up account traces back to it. The tool that is supposed to *test* your strategy is also the tool most eager to *flatter* it.

Overfitting (or curve-fitting) is what happens when your strategy stops learning the market and starts memorising the specific squiggles of the specific history you tested on. It nails the past because it was *shaped* to fit the past — the way a key is filed down to a single lock. Show it a new door and it does nothing.

Why more knobs = better backtest, worse live

Every parameter you add is a degree of freedom — another dial you can turn to bend the equity curve upward. With enough dials you can fit *anything*, including pure randomness. A rule with 8 tuned parameters over 2 years of daily bars has more knobs than it has meaningful, independent events to learn from. At that point you are not modelling the market; you are drawing a line through noise and calling it a signal.

• More parameters → more ways to accidentally match the noise in your sample.
• More tweaking → each "let me just adjust the stop" quietly fits one more historical accident.
• More metrics chased (Sharpe, drawdown, win-rate all maxed) → the result is squeezed so tight to the past that any deviation breaks it.

The cruel part is that the feedback loop feels like *progress*. You change a threshold from 20 to 22, the backtest return jumps, you feel smart. But you did not discover an edge — you found the number that best fit last year's luck. You are being paid in confidence and charged in future losses.

Watch it collapse: in-sample vs out-of-sample

Here is the picture that should be burned into your mind. A curve-fit strategy produces a gorgeous, smooth, up-and-to-the-right equity curve — on the data it was tuned on. The moment you run it on data it has never seen, the edge is gone and the curve does what noise does: wanders, then bleeds.

Manufacturing an edge out of pure noise

To really feel the danger, do the experiment that every quant should do once: build a "strategy" on data that has *no edge in it at all* — random numbers — and watch the parameter sweep hand you a winner anyway.

import numpy as np

rng = np.random.default_rng(class="n">0)

class="c"># Pure noise. There is NO signal in here to find. Returns average zero.
returns = rng.normal(class="n">0, class="n">0.01, size=class="n">1000)

best_sharpe = -np.inf
best_params = None

class="c"># Sweep class="n">400 combinations of two moving-average lengths.
for fast in range(class="n">2, class="n">22):
    for slow in range(fast + class="n">1, class="n">42):
        fast_ma = pd.Series(returns).rolling(fast).mean()
        slow_ma = pd.Series(returns).rolling(slow).mean()
        signal  = (fast_ma > slow_ma).astype(int).shift(class="n">1)   class="c"># long when fast>slow
        strat   = signal * returns
        sharpe  = strat.mean() / (strat.std() + 1e-class="n">9) * np.sqrt(class="n">252)
        if sharpe > best_sharpe:
            best_sharpe, best_params = sharpe, (fast, slow)

print(fclass="s">"Best Sharpe: {best_sharpe:.2f} with params {best_params}")
class="c"># -> class="s">"Best Sharpe: ~class="n">1.5 with params (x, y)"  ... on data that is class="n">100% noise.

Read that result again. There was no edge — the returns were random. Yet by testing 400 combinations and keeping the best, we "discovered" a strategy with a respectable Sharpe. That number is not skill; it is the maximum of 400 coin-flipping experiments. Search hard enough over noise and something always looks brilliant. This is why "I optimised the parameters and got a great backtest" is not evidence of anything on its own.

The rest of this part is the antidote. Every technique — out-of-sample testing, walk-forward, Monte Carlo, parameter-stability maps — exists for one purpose: to stop you from paying yourself in false confidence. You cannot eliminate overfitting, but you can measure it, and measuring it is what separates a trader from a gambler with a Python interpreter.