Optimization Targets and Search Engines

Every optimization has an objective, whether or not you chose one. AmiBroker will happily run several hundred backtests and hand you a list without ever asking what “best” means, and the two different things it does about that question — one for exhaustive runs, one for everything else — live in different places and behave differently. Getting this wrong is not a small error. The objective is the entire definition of what you are searching for.

By the end of this lesson you should be able to say where the Optimization target setting is and what it actually controls, choose a target using AmiBroker’s own metric names, predict the pathologies each common target invites, describe conceptually how a custom objective function is built, and name the three search engines AmiBroker ships with and the specific conditions under which a non-exhaustive search is the right choice.

The two different “what is best” mechanisms

An exhaustive optimization sorts its result list by Net % profit when it finishes. That is a presentation default, not a judgement, and it is not connected to any setting you chose. You re-sort by clicking a column header.

Separately, there is a field called Optimization target, and it lives in a place nobody guesses: Analysis window → Settings → Walk-Forward tab, at the bottom. Its default is CAR/MDD. It controls two things:

• which parameter set a smart (non-exhaustive) engine is searching for, and
• which in-sample winner a walk-forward run carries into each out-of-sample segment.

It does not re-sort an exhaustive result list.

✕Common mistake

Adding OptimizerSetEngine("cmae") to a formula, running it, and assuming the engine optimised the metric you had in mind. If you never set the Optimization target, the guide is explicit about what happens: it optimizes CAR/MDD — compound annual return divided by maximum percentage drawdown. That may well be a defensible objective. It is not one you chose, and you should not report it as though it were.

There is a naming wrinkle worth internalising now, because a typo here fails silently. AmiBroker uses two vocabularies for the same numbers: the HTML report row is labelled CAR/MaxDD, while the Optimization target field and the programmatic metric string read CAR/MDD. The same split applies to RAR/MaxDD versus RAR/MDD. Because the target is matched by name, copy the string from where you are typing it rather than from memory.

Choosing a built-in target

The target can be any built-in column, exactly as it appears in the optimization output. The candidates that come up most often, with AmiBroker’s own labels:

Target	What it rewards	The failure mode it invites
Net Profit %	Total money made over the whole test	Longer exposure and more capital at risk; ignores the path entirely
Annual Return %	Compounded annual return	The same, annualised — still says nothing about the drawdown you sat through
CAR/MaxDD	Return per unit of worst peak-to-trough decline	Sensitive to a single bad episode, which sets the denominator
RAR/MaxDD	The same, using exposure-adjusted return	Inherits the exposure problem below
Risk Adjusted Return %	Return divided by Exposure %	Explodes when the system is barely in the market; a rule that trades twice a decade can top this ranking
Profit Factor	Winners’ profit over losers’ loss	Blind to how many trades there were; superb values are routine on nine trades
Recovery Factor	Net profit over Max. system drawdown	One outstanding trade can carry it
Ulcer Performance Index	Return against the depth and duration of drawdown	Depends on a settable rate on the Report tab, so two machines can disagree
K-Ratio	Consistency of the equity line’s slope	Says nothing about the size of the return

Two structural points cut across the whole table.

Ratios with small denominators explode. Exposure % in the denominator of the risk-adjusted returns, and Max. system % drawdown in the denominator of CAR/MaxDD, are both quantities that some parameter combinations will make very small. The optimizer does not know that a 0.4% drawdown over eleven trades is an accident; it only knows the ratio is enormous.

Nothing in a built-in target knows about sample size. All trades — the report’s count of trades — is a column you can sort by, but no built-in target incorporates it. So whichever target you pick, the first thing to do with the result list is sort by All trades and see how thin the top of your ranking is.

∴Research note

Decide the target before you run, and write down why. A target chosen after seeing the result list is not an objective function; it is a way of describing whichever run you already liked. If you genuinely want to compare targets, that is fine — but do it as a stated comparison, and report all of them, not just the flattering one.

The target changes the winner

The table below is synthetic. I built six parameter sets with plausible-looking properties to show how the ranking moves; none of these figures came from a market, an instrument or a backtest, and none of them is a claim about anything.

Set	Net Profit %	CAR/MaxDD	Profit Factor	Exposure %	All trades
A	118	0.42	1.28	62	214
B	96	0.71	1.44	41	156
C	74	0.68	1.51	28	121
D	141	0.35	1.21	78	263
E	33	0.52	3.10	4	9
F	88	0.66	1.39	35	148

Rank by Net Profit % and D wins — it also carries the most exposure and trades most often, which is the same fact stated three ways. Rank by CAR/MaxDD and B wins. Rank by Profit Factor and E wins by a distance, on nine trades, which is not a result at all. Rank by return divided by Exposure % and E wins again, for the same reason: it is barely in the market.

Notice B, C and F. They sit close together on three of the five columns and their parameter values, if this were a real grid, would probably be adjacent. That cluster is the interesting object here — not the winner of any single ranking.

Custom objective functions, conceptually

When no built-in column expresses what you mean, AmiBroker lets you compute your own metric and use it as the target. The mechanism is the custom backtester, and the shape of it is:

Pseudocode — not valid AFL

in a custom backtest procedure, running in the portfolio phase:
    get the backtester object
    run the backtest
    read the built-in statistics you need
    combine them into your own number
    add that number to the report under a name you choose

Three things then have to be true together, or the target silently does not take effect:

1. the metric is added under an exact name by the custom backtest procedure;
2. the custom backtester is enabled;
3. that same name, spelled identically, is typed into the Optimization target field — the combo box does not enumerate custom metrics, so you type it.

The mechanics belong to the custom-backtester material rather than here; what matters at this point is knowing the facility exists, that it is where a sample-size floor or a drawdown-duration penalty would live, and that it comes with two documented limits. Custom metrics are unavailable under Individual Optimize, because that mode does not support the custom backtester at all. And a Monte Carlo percentile can serve as a target — the guide works through a CAR30/MDD example — but only if Monte Carlo is forced to run during optimization with SetOption( "MCEnable", 2 ), which the same page describes as strongly discouraged unless you are actually using a Monte Carlo metric as the target, because it can increase optimization time by orders of magnitude.

Exhaustive versus smart search

Leave the engine alone and AmiBroker performs an exhaustive full-grid search: every combination, every time. That is the default and, for two parameters at a sensible resolution, it is the right answer.

AmiBroker ships three non-exhaustive engines, selected in code — there is no combo box for this anywhere in the Settings dialog:

Fragment — not a complete formula

OptimizerSetEngine( "cmae" );        // or "spso", or "trib"
OptimizerSetOption( "Runs", 5 );

FastPeriod = Optimize( "Fast MA", 20, 5, 60, 1 );
SlowPeriod = Optimize( "Slow MA", 120, 60, 260, 1 );

Engine name	What it is	Documented defaults and notes
"spso"	Standard Particle Swarm Optimizer, based on SPSO2007	Supports Runs and MaxEval. The guide warns that picking good options can be tricky and results may vary considerably from case to case.
"trib"	TRIBES, an adaptive parameterless variant of PSO	MaxEval default 1000 (good for two or three dimensions), Runs default 5. The guide’s own verdict: performance is quite similar to PSO.
"cmae"	CMA-ES, covariance matrix adaptation evolutionary strategy	Runs default 5, MaxEval default 0 meaning “work it out for yourself” — the guide advises not setting it. Each run uses twice the population of the previous one, so keep Runs between 1 and 10.

All three are DLL plugins that ship with AmiBroker and come with full source in the ADK subfolder. They are first-party components, not downloads.

!Important

There is no genetic-algorithm engine in AmiBroker. Exactly three engines ship: SPSO, TRIBES and CMA-ES. Genetic algorithms appear in the guide only as background theory about the family of methods. If a forum post tells you to switch on AmiBroker’s GA optimizer, it is describing something that does not exist.

Runs and MaxEval are routinely confused. An evaluation is one backtest. A run is one complete execution of the algorithm, starting from a fresh random population, and it consumes many evaluations. Total work is roughly runs multiplied by evaluations per run. The guide’s reasoning for preferring several runs is worth keeping: five runs of a thousand evaluations is more likely to find the global optimum than one run of five thousand, because each restart begins somewhere new and can escape a different local optimum.

When a non-exhaustive search is appropriate

The honest answer is narrow.

Use it when the grid is genuinely unrunnable. The guide’s arithmetic: two parameters over 1..100 is 10,000 combinations, fine exhaustively; three is a million, still possible but lengthy; four is a hundred million; five is ten billion. Somewhere in there exhaustive search stops being an option and a sampled search is the only search available.

Use it when the parameters are continuous and the objective is reasonably smooth. That is the documented sweet spot. The guide’s caveat section is blunt about the opposite case: binary on/off parameters are not suited to any method that relies on the gradient of the objective function. If your system has several such switches, optimize the continuous parameters with an engine and change the switches by hand.

Note that step size is nearly free with CMA-ES. The guide states that decreasing the step argument does not significantly affect optimization time; what matters is the number of dimensions. Use a fine resolution and keep the parameter count low — that is the trade this engine offers.

And the reasons not to:

• You lose the surface. A sampled search does not evaluate a grid, so you cannot lay the result out as the surface the previous two lessons were built around. If your purpose is to see the shape of the parameter space — which, in this course, it usually is — an exhaustive run on a coarse grid is more informative than a smart run on a fine one.
• Results are not reproducible. Each run starts from a fresh random population, so two runs of the same formula on the same data can return different winners. That is a property to disclose in your notes, not a bug.
• No stochastic method promises the global optimum. The guide says so plainly, then makes an argument this course agrees with: if a single sharp peak exists somewhere in a vast space, missing it costs you nothing, because a lone peak is unstable and not replicable in real trading. The thing you are hunting is a plateau, and plateaus are what sampling methods find most easily.
• Progress estimates are approximate. CMA-ES skips evaluations when it detects convergence and can extend past its initial estimate, so the remaining-time display is described in the documentation as a best guess.

One more restriction to remember from the first lesson: Individual Optimize — the mode that spreads a single-symbol optimization across cores — supports exhaustive optimization only. The guide expects that limitation to persist. So “fast single-symbol optimization” and “smart engine” are mutually exclusive choices.

Recap

The Optimization target field sits on the Walk-Forward tab, defaults to CAR/MDD, and drives smart engines and walk-forward selection — not the sort order of an exhaustive result list, which arrives ordered by Net % profit regardless. Pick the target deliberately, spell it the way AmiBroker spells it in the place you are typing it, and know the pathology each one invites: exposure in a denominator, drawdown in a denominator, and trade count in nothing at all.

A different target picks a different winner from the same run, which is a good reason to decide before you look and to report the cluster rather than the champion. Custom objectives are possible through the custom backtester, need all three of their pieces in place, and are unavailable under Individual Optimize.

Exhaustive search is the default and usually the right tool at two parameters. SPSO, TRIBES and CMA-ES ship with AmiBroker for the cases where the grid cannot be enumerated; they trade away the surface and reproducibility for reach, and none of them promises to find the best cell — which matters less than it sounds, because the best cell was never the goal.

Check your understanding

Question 1. Where is the Optimization target setting, and what does it actually control?

• On the General tab; it sorts the optimization result list
• On the Walk-Forward tab; it drives smart engines and walk-forward in-sample selection, and does not re-sort an exhaustive result list
• In the 3D chart viewer; it selects the plotted metric
• In the Optimize button drop-down; it selects the metric for all optimizations

Show the answer and why

Answer: On the Walk-Forward tab; it drives smart engines and walk-forward in-sample selection, and does not re-sort an exhaustive result list

It is on the Walk-Forward tab even when you are not doing walk-forward, which is why first-time users of the smart engines are so often optimizing CAR/MDD without realising it — that is the default. An exhaustive list still arrives sorted by Net % profit; click a column header to change that.

Question 2. A parameter set tops the ranking on Profit Factor with a value above 3, and the All trades column shows 9. What is the reasonable reading?

• An unusually strong parameter set worth trading
• A sample too small to support the metric, and a target that has no notion of sample size
• Evidence that Profit Factor is the wrong metric in every situation
• Proof that the formula contains a look-ahead error

Show the answer and why

Answer: A sample too small to support the metric, and a target that has no notion of sample size

Profit Factor is winners’ profit divided by losers’ loss and knows nothing about how many trades produced it. Nine trades can produce almost any value. No built-in target incorporates sample size, which is why sorting the result list by All trades once, every time, is a standing habit rather than an optional check.

Question 3. Which statements about AmiBroker’s search engines are accurate? Select all that apply.

• Three engines ship: SPSO, TRIBES and CMA-ES
• A genetic algorithm engine is included and selectable
• Engines are selected only in AFL, with OptimizerSetEngine — there is no dialog for it
• Individual Optimize supports exhaustive optimization only

Show the answer and why

Answer: Three engines ship: SPSO, TRIBES and CMA-ES, Engines are selected only in AFL, with OptimizerSetEngine — there is no dialog for it, Individual Optimize supports exhaustive optimization only

Exactly three engines ship, all as DLL plugins with source in the ADK folder; genetic algorithms are discussed as background theory only. Engine choice is code-level, and Individual Optimize supports neither the smart engines nor the custom backtester — so the fast single-symbol mode and the sampled search cannot be combined.

Question 4. You have two parameters, each with about thirty sensible values, and your goal is to identify a robust region. Which approach fits the goal best?

• CMA-ES with a fine step, because it converges quickly
• An exhaustive grid, because you need every cell to lay out a surface
• TRIBES with Runs set to 10, to escape local optima
• An exhaustive grid on one parameter at a time, then combine the winners

Show the answer and why

Answer: An exhaustive grid, because you need every cell to lay out a surface

Nine hundred combinations is comfortably exhaustive, and a sampled search does not evaluate a grid, so it cannot give you the surface a region argument is built on. Optimizing one parameter at a time is worse still: it assumes the parameters do not interact, which is precisely what a surface would have shown you.

Check my answersClear