Skip to content
AmiBroker TA Course

Monte Carlo and Robustness

A backtest hands you one equity curve. One. Whatever it shows — a smooth climb, a sickening trough in the middle, a recovery at the end — happened exactly once, in one order, on one universe, over one stretch of history. You are then asked to make a decision that depends on the answer to a question the backtest never addressed: how different could it have been?

That is the question this part is about, and it is worth being precise about how small the answerable portion of it is. Resampling a set of trades tells you something about how sensitive your result was to the order and the selection of the trades you actually took. It tells you nothing whatsoever about the trades you did not take, the markets you did not test, the costs you did not model, or the regime that has not happened yet. Most of the confidence people draw from a Monte Carlo page is confidence in the wrong direction.

What is actually here

Section titled “What is actually here”

AmiBroker ships a Monte Carlo simulator. It is documented, it is fast, and its scope is narrower than its reputation. It bootstraps — samples with replacement — either the realised trade list or the bar-by-bar percentage changes in portfolio equity, replays the resampled series a thousand or more times, and reports the distribution of final equity, annual return, drawdown and lowest equity that results.

That is the complete list. It does not randomise prices. It does not randomise your parameters. It does not generate synthetic price series, invent slippage, or simulate a market crash you never had data for. Anything you have read that attributes those capabilities to AmiBroker is describing a different product. This part sticks to what the User’s Guide documents, names the AmiBroker settings by their exact labels, and is explicit about where the built-in feature stops and where you would have to write your own code.

The assumption everything rests on

Section titled “The assumption everything rests on”

Bootstrapping is valid when the things you are resampling are exchangeable — when the order they arrived in carried no information, so that any reordering is as plausible as the one that happened.

Trades are not exchangeable. They cluster. A trend-following system makes most of its money in a handful of sustained advances and gives a slice of it back in every choppy stretch, and those stretches are not sprinkled evenly through history; they arrive in blocks, because market conditions arrive in blocks. Losing trades keep company with other losing trades. When you shuffle them apart, you construct a world with tamer drawdowns than the one you are going to trade in.

This is not a technicality to note and move past. It is the reason the drawdown percentiles on the Monte Carlo page tend to be optimistic, and the first lesson gives you a formula that measures the clustering in your own results rather than asking you to take the point on trust.

What you will be able to do

Section titled “What you will be able to do”

By the end of this part you should be able to open the Monte Carlo tab and say what each setting changes; choose between the trade-list and equity-changes sampling modes for a reason rather than by default; read the percentile table in the direction it is actually written, including the sign switch that reverses the meaning of the drawdown column; explain why the green and red lines on the equity chart are not a best and worst case; distinguish sequence risk from terminal risk and know when each one applies; run a ruin simulation and understand which of its assumptions are flattering you; and state, in one paragraph, what a robustness test does and does not license you to conclude.

You should also come out of it more sceptical than you went in. That is deliberate. A Monte Carlo page adds decimal places to a backtest; it adds no new information about the market. If the backtest that fed it contained look-ahead bias, survivorship bias, optimistic fills or parameters chosen on the same data, the simulation will restate all of it with five percentile columns and a pair of nicely drawn distribution charts. The final lesson is about exactly that.

What you need

Section titled “What you need”

Any edition of AmiBroker will do — the User’s Guide places no edition restriction on the Monte Carlo simulator, and the practical difference between Standard and Professional here is throughput rather than capability. End-of-day data is enough; nothing in this part needs a live feed or a paid subscription. You will need one completed portfolio backtest to work on, and a formula is supplied if you do not already have one from Part 28.

You should have Parts 28 to 32 behind you. This part assumes you can read AmiBroker’s backtest report in its own vocabulary, that you know what look-ahead and survivorship bias look like, and that you have already met the difference between an in-sample result and an out-of-sample one. Robustness testing is the last thing you do to a candidate system, not the first.

Part33 of 36Level4 — Trading System ResearcherPages4Estimated time1.8 hours

0 / 4 lessons in this part completed