Home › Resources › Backtest Overfitting
How to Know If Your Backtest Is Overfit
A backtest that returns 200% a year with a smooth equity curve feels like proof. It isn't. It's one outcome — the specific sequence of trades your strategy happened to produce on the specific historical data you tested it against. Pay a prop firm eval fee on the strength of that one outcome alone, and you're betting the backtest wasn't just a lucky roll. Here's how to actually check.
Why a perfect backtest can still be a false edge
Every backtest is fit to one path through history. If you adjusted parameters — moving average length, stop distance, entry filter thresholds — until the equity curve looked good, you didn't discover an edge. You found the specific combination of numbers that happened to work on that specific data. This is curve-fitting, and it's the single most common reason a strategy that backtests beautifully fails immediately on a live prop firm evaluation, where every trade is on data the strategy has never seen.
The tell isn't how good the backtest looks. It's whether the result survives being tested against alternate versions of history that could plausibly have happened instead.
The sample size problem
A strategy with 30 trades and a 70% win rate sounds strong. It isn't a reliable signal yet — with that few trades, a handful of results going the other way can swing the win rate by 15-20 points. Treat under 100 trades as not enough to conclude anything. 200+ trades is a reasonable floor before you trust a profit factor or win rate as a real, repeatable number rather than sampling noise.
Monte Carlo validation: testing more than one version of history
Your backtest produced one equity curve because your trades happened in one specific order, on one specific slice of history. Monte Carlo validation asks a different question: if the same set of trades had happened in a different order — or if history had been slightly different — would the strategy still survive?
Two practical methods:
- Trade-order reshuffling. Take your actual sequence of trade returns and randomly reshuffle the order thousands of times. Each reshuffle produces a different equity curve and a different maximum drawdown, even though it's the exact same set of trades. If a large share of reshuffled paths blow through your prop firm's drawdown limit even though your original backtest never did, the original result was a favorable ordering, not a robust edge.
- Bootstrap resampling. Randomly resample trades with replacement (the same trade can be picked more than once) to build thousands of synthetic equity curves of the same length as your original. This tests sensitivity to the exact trade sample, not just its order. Look at the 5th-percentile outcome across all resamples — that's a far more honest worst case than the single drawdown number your original backtest reported.
A strategy that passes both — survivable drawdown and positive expectancy across the large majority of resampled paths — has evidence of a real edge. A strategy where a shuffled order or resample flips the result negative was likely riding a specific favorable sequence, not a repeatable one.
Walk-forward validation: the out-of-sample test
Split your historical data into two blocks. Optimize parameters only on the first block (in-sample). Then run the strategy, unchanged, on the second block (out-of-sample) — data the optimization never saw. If performance collapses on the out-of-sample block, the strategy was fit to noise specific to the in-sample period, not to a persistent market behavior.
A useful sanity metric here is Walk-Forward Efficiency: out-of-sample performance divided by in-sample performance. A ratio near 1.0 means the strategy holds up on unseen data. A ratio near zero or negative means the in-sample result doesn't generalize — exactly the failure mode that shows up as "great backtest, blown eval."
A pre-eval checklist
| Check | What it catches |
|---|---|
| 200+ trades in the backtest | Small-sample luck masquerading as edge |
| Bootstrap resample — 5th percentile still profitable | A result that depends on one lucky trade sequence |
| Reshuffled-order max drawdown stays inside the firm's limit | A drawdown number that was itself a favorable ordering |
| Out-of-sample block performance close to in-sample | Curve-fitting to one specific historical period |
| Realistic slippage and commission applied | Edge that only exists at zero-cost fills |
| Tested across both trending and ranging periods | A strategy that only works in one market regime |
What this looks like in Pine Script
TradingView's Strategy Tester doesn't run Monte Carlo resampling natively, but you can export your closed-trade list (List of Trades tab → export) and run the reshuffle/bootstrap tests in a spreadsheet or a short Python script. The Pine Script side of the job is making sure the exported trade list is actually representative of live conditions:
// ── Realistic cost assumptions before you trust the export ─────────
// Set these in Strategy Properties before generating your trade list,
// not left at TradingView's zero-cost defaults.
// Properties tab:
// Slippage: 2-3 ticks (MES/MNQ), 3-5 ticks (CL/GC on fast bars)
// Commission: your actual broker's per-contract round-turn cost
// Verify "Recalculate After Order Filled" is OFF unless your
// live execution actually recalculates mid-bar
// A trade list exported with zero slippage/commission is not
// the same distribution you'll trade live — resample that one
// and you're just validating a fantasy.Want someone else to run the check?
The Strategy Audit reviews your Pine Script for exactly this class of issue — sample size, overfitting risk, and prop firm rule compliance — with a written report in 48 hours.
Get a Strategy Audit — $79Or start from a strategy already built around prop firm drawdown rules