Walk-Forward Methodology

A single holdout gives you one out-of-sample observation, spent once. Walk-forward analysis gives you a sequence of them, at the price of answering a narrower question. The narrowing is the interesting part, and it is the part most descriptions skip.

By the end of this lesson you should be able to draw both window layouts from memory, say what assumption each one encodes about how markets change, size the windows from your trade rate rather than from a round number of years, and state precisely which claims a positive walk-forward result does and does not support.

What is actually being simulated

The procedure is a loop. Optimise the parameters on a past segment of data — the in-sample segment. Take the winning parameter set and backtest it, unchanged, on the segment that immediately follows — the out-of-sample segment. Move both windows forward and repeat until you run out of history. Judge the system on the out-of-sample segments only.

AmiBroker’s User’s Guide states the purpose without hedging: the point is to determine whether the performance of an optimised system is realistic or the result of curve-fitting, and out-of-sample performance is “a realistic estimate of how the system would work in real trading”. It then adds the sentence that most people quote and few act on: in-sample results do not matter much, because they are always good.

The important shift is in what is being tested. You are no longer testing “a 55-bar breakout with a 20-bar exit”. You are testing “re-optimise the breakout and exit lengths every year, on the previous four years, choosing by CAR/MDD”. The deliverable is the procedure, not the parameters. If you run a walk-forward test and then trade a single fixed parameter set chosen from the whole history, you have not traded the thing you tested.

!Important

The premise being assumed is that the recent past is a better guide to the immediate future than the distant past. The User’s Guide says so explicitly, and immediately warns that this “may or may not be the case as markets go through bear/bull cycles”. Walk-forward does not establish that premise. It assumes it, and everything downstream inherits the assumption.

Anchored windows

In an anchored walk-forward, the in-sample window’s start date stays fixed while its end date moves forward. The training set expands: every step uses all history from the beginning up to the current point.

Anchored (expanding) walk-forward

Step 1

In-sampleOOS

Step 2

In-sampleOOS

Step 3

In-sampleOOS

Step 4

In-sampleOOS

2008201320182023

The start date never moves. Each step trains on everything that came before it.

The assumption this encodes is that whatever edge exists is stable, so more data is better data. Its practical virtues are that later steps are estimated from large samples, and that the parameter choices tend to settle down and stop jumping about.

Its weakness is the mirror image. If the market’s behaviour genuinely changed in 2015, an anchored window in 2024 is still being partly governed by 2008, and the weight of old data grows with every step, so the procedure adapts more and more slowly precisely as it accrues more history. The early steps also differ in kind from the late ones: step 1 might be fitted to four years and step 16 to nineteen, so the out-of-sample segments are not really comparable observations of the same procedure.

Rolling windows

In a rolling (non-anchored) walk-forward, the in-sample window has a fixed length and both its ends move forward together. Old data drops out of the back as new data enters the front.

Rolling (non-anchored) walk-forward

Step 1

In-sampleOOS

Step 2

In-sampleOOS

Step 3

In-sampleOOS

Step 4

In-sampleOOS

2008201320182023

A fixed-length training window slides forward. Every step is estimated from the same amount of data.

This encodes the opposite assumption: that market behaviour drifts, and that data older than the window length is more misleading than helpful. Every step is estimated from an identical quantity of data, which makes the out-of-sample segments genuinely comparable, and the procedure responds to change at a rate you chose deliberately.

The cost is estimation noise. A short window contains few trades, so the “winning” parameter set on each step is chosen from a noisier surface, and the selected parameters can jump violently from step to step. Watching those jumps is itself diagnostic: a procedure whose chosen breakout length oscillates between 20 and 100 across consecutive steps is not adapting, it is chasing noise.

	Anchored	Rolling
In-sample start	Fixed	Moves with the step
In-sample length	Grows every step	Constant
Assumes	The edge is stable; more data is better	Behaviour drifts; recent data is more relevant
Adaptation speed	Slows as history accumulates	Constant, and chosen by you
Estimation noise	Falls over time	Constant, and higher for short windows
Steps comparable?	Not really — later steps use more data	Yes
Typical failure	Too slow to notice a regime change	Chases noise, parameters oscillate

Neither is correct in general. Choose the one whose assumption you are prepared to defend in writing, and note that choosing between them by which produced the better out-of-sample result converts your out-of-sample data into in-sample data.

Sizing the windows

Three numbers define the geometry: the in-sample length, the out-of-sample length, and the step. In AmiBroker’s Easy mode the last two are the same number and the out-of-sample segment follows the in-sample segment immediately, which removes the two commonest ways of building an invalid procedure. Take that constraint as a gift rather than a limitation.

Size the in-sample window by trades, not by years. An optimisation is an estimate, and the precision of an estimate depends on the sample it is computed from. If your system takes about 25 trades a year across the universe, a two-year in-sample window offers roughly 50 trades from which to choose between 153 parameter combinations. That is not estimation, it is selection from noise. Work out your approximate trade rate first, decide how many trades you need before a difference between parameter sets means anything, and derive the window length from that.

Size the out-of-sample window by how often you are willing to re-optimise. The step is a real operational decision: it is how often you would sit down and re-run the optimisation in practice. Monthly re-estimation on daily bars usually means each out-of-sample segment contains a handful of trades and the procedure spends its life reacting to the last few weeks. Annual re-estimation gives segments substantial enough to read, at the cost of adapting slowly.

Then count your steps. The arithmetic is unavoidable:

Pseudocode — not valid AFL

steps = ( total_history - in_sample_length ) / step_length
total_out_of_sample = steps * out_of_sample_length

Twenty years of history, a four-year in-sample window and a one-year step gives sixteen steps and sixteen years of out-of-sample data. Shrinking the in-sample window to two years gives eighteen steps. Extending the step to two years halves the step count to eight. The figures are pure arithmetic about your date range; they say nothing about any strategy.

And count your run time. A walk-forward run is a sequence of complete optimizations: sixteen steps at 153 combinations each is 2,448 full portfolio backtests, and each of those sweeps your whole watch list. On the Standard edition you get 2 threads per Analysis window; the Professional edition allows up to 32. Multiply before you press the button, not after.

How many steps do you need

The instinct is to ask for a number of steps. The better question is how many out-of-sample trades the run produces in total, because that is what the statistics are computed from. Sixteen steps of four trades apiece is 64 trades — a thin sample whichever way you slice it, and one where a single outlier can carry the result.

Two further cautions. First, out-of-sample segments are not independent observations. They come from one market history, they are frequently produced by overlapping in-sample windows, and the same few dominant market episodes run through all of them. Treating sixteen steps as sixteen independent trials overstates your evidence considerably.

Second, more steps is not straightforwardly better. Steps are bought by shortening the in-sample window, the step, or both; each shortening makes the per-step estimate noisier. There is a genuine trade-off and no formula that resolves it, which is why the honest thing to do is to report the geometry alongside the result and let a reader judge.

Read the parameters the steps chose

The out-of-sample equity curve is the headline, but the most informative column in a walk-forward result is often the list of parameter values each in-sample optimisation selected. Three patterns recur, and they mean different things.

Values that stay in a neighbourhood. Successive steps pick 45, 50, 45, 55, 50. This is what you hope for: the objective surface has a stable region and the optimiser keeps landing in it. The parameter is doing something.

Values that oscillate wildly. Steps pick 20, 95, 25, 100, 30. The optimiser is choosing between combinations that are barely distinguishable on the evidence available, and small amounts of noise are flipping the winner from one extreme of the range to the other. The honest reading is that the in-sample window is too short, or the parameter has little effect, or both. Either way the “adaptation” is fictional.

Values pinned to an end of the range. Every step chooses 100 when the range stops at 100. The optimum is probably outside the range you declared, and you are measuring the edge of your own grid rather than a property of the market. Widen the range and re-run — but note that you have now made a decision using out-of-sample information, and record it.

There is a fourth case worth naming because it flatters: values that move slowly and monotonically, say 30, 35, 40, 45, 50 across five steps. That looks like graceful adaptation and is often something duller — a parameter tracking a slow change in volatility or in the composition of your universe. It is a hypothesis to test, not a finding.

What walk-forward does and does not establish

It does establish what a specified re-estimation procedure would have produced on this data, over this universe, with these costs and these fills — with each parameter choice made using only information available at the time. That is a genuine and useful thing, and it is strictly more than a single optimised backtest tells you.

It does not establish that the strategy has an edge. A positive out-of-sample result is consistent with an edge, and also with a favourable market period, with a lucky draw from a small sample, and with any bias that runs through your whole database, because survivorship bias and look-ahead bias do not respect segment boundaries. Walk-forward tests the parameter selection; it cannot repair a contaminated universe or a formula that reads tomorrow’s bar.

It does not remove the multiple-comparisons problem. Walk-forward twenty ideas, pick the one with the best out-of-sample composite, and that composite is now an in-sample statistic: you selected on it. The discipline that applies to parameters applies with equal force to strategies, and it is much less often observed.

Its own settings are parameters. In-sample length, step, anchored or rolling, and the optimisation target are four more choices. Trying several combinations and keeping the one with the best out-of-sample result is overfitting one level up, and it is harder to spot because it happens in a dialog box rather than in a formula.

?Reality check

The claim “if a system passes walk-forward, it is safe to trade” is worth holding up to the light. A walk-forward pass is one piece of evidence about parameter stability on one history. It says nothing about whether your fills were achievable, whether your universe was knowable at the time, whether the market regime that generated the result persists, or whether you can sit through the drawdown. Treat a pass as permission to continue investigating, not as a verdict.

What changed

You should now think of a walk-forward result as the output of a procedure you could write down as an operating instruction — “every January, re-optimise these two parameters on the last four years, ranked by CAR/MDD, and trade the winner for the year” — and you should be able to defend the four geometry choices behind it. You should also be clear that the sequence of out-of-sample segments is a set of correlated, sometimes tiny samples rather than a series of independent trials.

The next lesson turns this into the actual settings dialog, where the geometry becomes six fields and one combo box, and where the Preview list will tell you exactly what you have built before you spend an evening running it.

Check your understanding

Question 1. A walk-forward run with a rolling four-year in-sample window and a one-year step produces a good out-of-sample composite. What have you learned you can trade?

• The parameter set that won most often across the steps
• The parameter set from the final step, which used the most recent data
• The re-estimation procedure itself: re-optimise annually on the last four years
• Any parameter set inside the plateau the optimisation found

Show the answer and why

Answer: The re-estimation procedure itself: re-optimise annually on the last four years

The out-of-sample result is the output of the whole loop. Trading one fixed parameter set drawn from the run is trading something that was never tested.

Question 2. Which assumption does an anchored walk-forward encode?

• That market behaviour drifts, so old data is misleading
• That whatever edge exists is stable, so more history improves the estimate
• That the out-of-sample window must equal the step length
• That parameters should be re-estimated as often as possible

Show the answer and why

Answer: That whatever edge exists is stable, so more history improves the estimate

An expanding training window says that data from fifteen years ago is still informative. That is a claim about the market, and it is the opposite of the claim a rolling window makes.

Question 3. Your database spans 2004-2024. You use a rolling five-year in-sample window and a two-year step. Roughly how many walk-forward steps does that produce?

• Four
• Eight
• Ten
• Twenty

Show the answer and why

Answer: Eight

Twenty years of history minus a five-year training window leaves fifteen years to walk through, and a two-year step covers that in about eight moves. The arithmetic is a property of your date range, not of the strategy.

Question 4. Which of these would turn your out-of-sample composite back into an in-sample statistic? Select all that apply.

• Trying three in-sample window lengths and keeping the one with the best out-of-sample result
• Walk-forwarding six strategies and presenting the best one
• Changing the optimisation target from CAR/MDD to Net Profit % after seeing a poor result
• Reporting the number of out-of-sample trades alongside the composite

Show the answer and why

Answer: Trying three in-sample window lengths and keeping the one with the best out-of-sample result, Walk-forwarding six strategies and presenting the best one, Changing the optimisation target from CAR/MDD to Net Profit % after seeing a poor result

Each of the first three selects on the out-of-sample outcome, which is exactly what makes data in-sample. Reporting the trade count adds information for the reader and selects nothing.

Check my answersClear