Journalling and Separating Two Kinds of Performance

“The system stopped working” is not a diagnosis. It is at least three different situations wearing one sentence: the rules were never as good as the backtest suggested; the rules were fine and were not followed; or nothing at all went wrong and a normal losing run was mistaken for evidence. Telling them apart requires two separate measurements, and almost nobody takes the second one.

By the end of this lesson you should be able to specify exactly what a journal row contains and why every field must be knowable before the outcome is, measure strategy performance using AmiBroker’s own vocabulary, define and measure your own execution performance as a distinct quantity, compute the gap between the two, run a review schedule that permits a system change at one point and forbids it at every other, and recognise the situations in which the honest conclusion is that nothing should change.

Two measurements, not one

Two records, one comparison

1. 1Rules as writtenBacktest over the same period, universe and costs
2. 2Rules as executedThe journal: what you actually did, decision by decision
3. 3Paired comparisonMatched at the decision, not at the summary
4. 4Located verdictStrategy, execution, or neither

Each measurement is meaningless on its own; the useful quantity is the difference between them.

Strategy performance is what the rules would have produced over a period, measured by the backtester. It is a property of the rules, the universe, the costs and the period. It has nothing to do with you.

Trader execution performance is the difference between what the rules said and what you did. It is a property of you, your process and your circumstances. It has nothing to do with whether the rules were any good.

Neither of them is your account statement. Your account is the composition of the two, plus whatever the market did, and reading it as a measurement of either one is the mistake this whole lesson exists to prevent.

What to record per decision

The journal is written at the moment of decision, and it obeys one rule: every field must be knowable before the outcome is. A journal that mixes pre-decision fields with post-hoc commentary is not a record, it is a memoir, and the hindsight leaks in through the commentary.

The practical shape is two blocks of fields in one row.

Written by the exploration, at signal time:

Field	Why it is there
SignalDate	The bar the rules fired on, in ISO form so it sorts and parses
Ticker	The instrument, in the database’s own spelling
RuleVersion	Which version of the rules produced this. Without it, a rule change silently rewrites the meaning of every earlier row
SignalClose	The price the rules saw
PlannedStop and StopDistance	The risk unit. Every later deviation is best expressed as a fraction of this
PlannedShares, PlannedValue, PlannedRisk	What the sizing rule said, before you touched it
ParticipationPct	Whether the plan was deliverable at all
EntryRule	The fill convention assumed — “next open” — because the actual entry price does not exist yet

Written by you, after the fact:

Field	Why it is there
Taken	take, skip or defer — the three-word outcome from the review protocol
ActualEntry, ActualShares, ActualStop	What actually happened, in the same units
DeviationReason	A category, not a sentence: structure, stop too wide, concentration, event, data, size not viable, hesitation, override
ReviewerNote	The one place prose is allowed

!Important

The rows for candidates you skipped are the most valuable rows in the file, and they are the ones everyone omits. A journal of trades taken can only tell you about trades taken. The gap between the strategy and you lives largely in the signals that never became positions, and if those rows are missing, that gap is unmeasurable in principle rather than merely inconvenient.

Strategy performance, in AmiBroker’s vocabulary

Strategy performance is measured by running the rules over the period, in the backtester, with the same universe, the same costs and the same execution assumptions as the live process. Use AmiBroker’s own labels; the trading-blog vocabulary and the report do not use the same words for the same numbers, and the mismatches are not cosmetic.

Report label	What it is	Trap
Net Profit / Net Profit %	Total profit, in currency and as a percentage of initial capital	Net Profit % is a total return, not annualised. In the portfolio backtester it includes positions still open at the end, closed at the last close
Annual Return %	Compounded annual return	The report row is not called “CAGR” and not “CAR”; CAR is the metric string. The annualisation exponent uses 365 calendar days between the first and last bar
Exposure %	Value-weighted market exposure, bar by bar	A low exposure inflates both risk-adjusted return figures dramatically
Risk Adjusted Return %	Annual Return % divided by Exposure %	“Risk adjusted” here means exposure-adjusted only. It has nothing to do with volatility or drawdown
Net Risk Adjusted Return %	Net Profit % divided by Exposure %	Built on the total return, not the annualised one. Constantly confused with the row above
Max. system drawdown / Max. system % drawdown	Largest peak-to-valley decline in portfolio equity	The percentage figure is reported as a negative number. There is no report row called “Max Drawdown”
Max. trade drawdown / Max. trade % drawdown	Largest decline within a single trade	In the portfolio backtester the percentage is based on the trade’s value at entry, not on total equity
Recovery Factor	Net Profit divided by Max. system drawdown	Divides by the currency drawdown, unlike the two ratios below
CAR/MaxDD	Annual return divided by the percentage drawdown	The report writes CAR/MaxDD; the metric string and the walk-forward optimisation target read CAR/MDD
RAR/MaxDD	Risk-adjusted return divided by the percentage drawdown	Inherits the exposure sensitivity of RAR
Profit Factor	Profit of winners over loss of losers	Undefined with no losers, and spectacular on tiny samples
Payoff Ratio	Average win over average loss	Says nothing about profitability until read with the winners’ percentage
Avg. Profit/Loss / Avg. Profit/Loss %	Average per trade, in currency and percent	The documentation notes these are “also known as” Expectancy. There is no report row literally called Expectancy
Avg. Bars Held	Mean bars in trade	Enter today, exit tomorrow counts as two bars, not one day
Standard Error	Choppiness of the equity line about its regression	In equity units, so not comparable across tests of different size or length
Risk-Reward Ratio	Equity-line slope divided by its standard error	Not the per-trade stop-versus-target ratio. Completely different quantity, same everyday phrase
Ulcer Index / Ulcer Performance Index	Drawdown depth and duration, and a return measure built on it	Both move when the rate settings on the Report tab are changed
Sharpe Ratio of trades	Computed from the distribution of individual trade returns, annualised by bars per year over average bars per trade	Not comparable with a fund’s Sharpe ratio, which is computed from periodic equity returns. The label says “of trades” for a reason
K-Ratio	Consistency of returns from the equity-line regression	Depends on bar count, so not comparable across tests of different lengths

Two practical consequences follow.

Two reports are only comparable if their settings match. Periodicity, commission settings, the two rate settings on the Report tab that feed Sharpe Ratio of trades and Ulcer Performance Index, and the choice of engine all change the numbers. The report can embed the formula and the settings that produced it; turn that on, and store the reports, so that a comparison months later is a comparison rather than an act of faith.

The report has no commissions row. The old backtester had Total commissions paid; the portfolio backtester’s documented metric list does not. If your evaluation needs total costs, add them yourself as a custom metric from a custom backtest procedure — the mechanism Part 36 introduces.

∴Research note

Record a small, fixed set of metrics every quarter, chosen in advance, and never expand the set after seeing the results. Six is plenty: Annual Return %, Exposure %, Max. system % drawdown, CAR/MaxDD, Profit Factor and the number of trades. Choosing which statistics to report after looking at them is data snooping applied to your own record.

Trader execution performance

This is your own measurement, and it needs saying plainly: AmiBroker does not report any of what follows. These are quantities you compute from the journal, and the names below are this course’s names, not AmiBroker’s.

Five categories of deviation cover almost everything:

1. Omission. A signal fired and no position was taken. Measured as signal coverage: decisions taken divided by signals produced, over a period.
2. Addition. A position was taken with no signal behind it. Measured as a count, and as a share of total positions. This is the category people are least willing to record and the one that most often explains a divergence.
3. Timing and price. The position was taken, at a different price or on a different day than the rules assumed. Best expressed in R units — the deviation divided by the planned stop distance — because that makes it comparable across instruments and across time.
4. Size. Size ratio: actual shares divided by planned shares. A distribution centred on 1.0 with small spread is a process being followed; a bimodal distribution usually means “full size when confident, half size when not”, which is a discretionary overlay you have added without testing it.
5. Exit deviation. The rules said exit and you did not, or they did not and you did. Measured the same way: in R units, and in bars.

None of these has a target value except by comparison with itself over time. A signal coverage of 0.6 is not a failing grade; it is a fact that has to be paired with the outcome of the 40 per cent you skipped.

✕Common mistake

The seductive error here is to look only at the skipped signals that went on to be losers and conclude that your judgement is adding value. Recording only the skips you remember is a selection effect. This is exactly why the journal must contain a row for every signal, written before the outcome, with the reason categorised at the time — the discipline exists to stop the sample from being chosen after the fact.

Measuring the gap

The gap is measured with a shadow backtest: run the rules over precisely the period the journal covers, on the same universe, with the same costs and the same fill assumptions, and compare it with your record. Three levels of comparison, in increasing order of usefulness.

Level 1: the summary. The strategy’s Net Profit % over the period against your actual return over the period. This tells you the size of the gap and nothing about its cause. It is worth computing because it sets the scale of the question.

Level 2: the decomposition. Split your journal into the five categories above and attribute the gap. In R units this is additive and legible: omission cost 4.2R, size deviation cost 1.1R, entry timing cost 0.4R, additions cost 3.0R, exits gained 0.8R. Now the conversation is about something specific.

Level 3: the pairing. Match each backtest trade to a journal row by ticker and signal date. Every unmatched row on either side is itself a finding: a backtest trade with no journal row means the operating loop missed a signal, which is a process defect rather than a judgement one; a journal row with no backtest trade means either an addition or a difference between the live formula and the one you backtested, and the second of those is worth finding out about immediately.

∴Research note

The pairing has a prerequisite most people discover too late: the shadow backtest must use the same rule version as the journal rows it is being compared against. This is the entire reason RuleVersion is a column. Comparing three months of journal rows against a backtest of the rules as they stand today measures the rule change, not the execution.

Two results are worth naming in advance.

A large gap with a good strategy is a process problem, and process problems have process fixes: a shorter candidate list, a review protocol you can actually complete, a sizing rule with no discretionary step in it, orders placed at a fixed time.

A near-zero gap with a poor strategy is the pleasant case, in a bleak sort of way. You are executing faithfully and the rules are the problem, which means the research loop has somewhere useful to go, and none of the effort spent on discipline was wasted.

Reviewing on a schedule

The schedule matters as much as the measurements, because the main function of a schedule is to say when you may not change things.

Cadence	Duration	What it looks at	What it may change
Daily, after the process	5 min	Was the process followed? Were all rows journalled?	Nothing about the rules. Only process defects — a missing export, a step skipped
Weekly	20–30 min	Execution deviations for the week, by category. Open positions against their stops	Process and habits. Never a rule
Quarterly	Half a day	Shadow backtest for the quarter; the six metrics; the gap decomposition; the research backlog	Rules — this is the only slot where a rule may change
Annually	A day	Universe specification, cost assumptions, regime definition, whether the whole approach is still viable	Anything, including the decision to stop

!Important

The quarterly review is the only place a rule may change, and that constraint is doing more work than any individual technique in this part. Its purpose is not caution for its own sake. It is that a rule changed in reaction to a recent outcome cannot be evaluated, because the record before and after the change are no longer measurements of the same thing.

When to change the system, and when not to

Legitimate reasons to change

• A specification defect. The rules do something you did not intend — a stop that cannot trigger, a filter that excludes what it was meant to include. Not a performance question at all; fix it, note the version, and treat every earlier result as belonging to the old version.
• A broken assumption. Commission structure changed, the liquidity of your universe changed, the data source changed, an exchange changed its session. The rules were correct for a world that no longer exists.
• A discovered bias. You find that the universe carried survivorship, or that a value was available in the formula before it was available in reality. The old results are not evidence and the change is not optional.
• An operational impossibility. The planned sizes are consistently undeliverable at the participation cap. The rules are fine and cannot be run; something must give.
• Evidence from the research loop. A new hypothesis has been through the whole procedure — in-sample, out-of-sample, walk-forward, robustness — and survived. This is the slow, expensive, legitimate path.

Illegitimate reasons to change

• A drawdown inside the range the evidence already showed. If the backtest’s Max. system % drawdown was −24% and you are 15% down, nothing has happened that the evidence did not predict. The Monte Carlo percentile table from Part 33 exists partly to give you this number before you need it.
• A losing streak shorter than the historical maximum. "LosersMaxConsecutive" is in the report for a reason. A run of six losses in a system whose backtest contained a run of eleven is not information.
• A parameter that looks better in hindsight. It always exists. Finding it costs nothing and proves nothing.
• Boredom, or a quiet regime. A gated system is inactive for long stretches by construction. Inactivity is the system working, and the urge to loosen a filter during it is the single most reliable way to undo a year of careful work.
• Somebody else’s result. You have no access to their universe, period, costs or assumptions.

The honest answer is often “change nothing”

“Change nothing” is a decision, and it goes in the journal with its reasoning, because otherwise next quarter you will not remember whether you considered the question. Write it as a claim that could be checked: drawdown of 15% is within the −24% the backtest showed and the 5th percentile of the Monte Carlo table; the losing run of six is shorter than the eleven in the test; execution gap for the quarter was 0.3R; no change.

There is a sample-size point underneath all of this. A quarter of a daily end-of-day system might contain twenty or thirty trades. Twenty trades tell you very little about a distribution whose behaviour is dominated by its tails, and the shorter the record, the more of what you are looking at is noise. That is not a reason to stop measuring. It is a reason to make the measurements you take precise, comparable and durable, so that in three years there is a record long enough to say something.

The journal export

Goal

Produce the machine-readable half of the journal automatically, at signal time, so that the pre-decision fields cannot be reconstructed later from memory — and reserve the post-decision fields as empty columns so the two halves live in one row.

Complete formula

Complete runnable AFL

decision-journal-export.afl

// decision-journal-export.afl
// Part 35 - Journalling and Separating Two Kinds of Performance
//
// Writes the machine-readable half of the decision journal: one row per
// candidate, containing every field that is knowable BEFORE the outcome is,
// plus empty columns for the fields only a human can fill in afterwards.
//
// The split matters more than the formula does. Everything this formula emits
// is what the rules said. Everything in the empty columns is what you did.
// Keeping them in one row, written at one moment, is what later makes the gap
// between the two measurable instead of a matter of recollection.
//
// The default columns are switched off so that File -> Export HTML/CSV
// produces a file whose columns line up with a journal spreadsheet. The ticker
// is emitted explicitly instead, because a journal row without a symbol in a
// known column position is not machine-readable.
//
// Run settings this formula assumes:
//   Apply to: the watch list that defines your universe
//   Range:    All quotations, with Filter restricting output to the last bar
//   Then:     File -> Export HTML/CSV, with the New Analysis window focused
//
// Assumptions:
//   - Written on the evening of the signal bar, before any order exists. The
//     planned-entry column is therefore a rule, not a price.
//   - The parameters below must be the same ones the candidate sheet used. If
//     they are not, the journal records a plan you never actually saw.
//   - Exported columns follow the displayed sort order, so changing the sort
//     changes the file. Sort once, deliberately, and leave it alone.

_SECTION_BEGIN("Decision journal export");

AccountEquity   = Param( "Account equity", 100000, 1000, 100000000, 1000 );
RiskPercent     = Param( "Risk budget per trade (% of equity)", 0.5, 0.05, 5, 0.05 );
MaxPositionPct  = Param( "Position value cap (% of equity)", 10, 1, 100, 1 );
MinTurnover     = Param( "Liquidity floor: average daily turnover", 2000000, 0, 100000000, 100000 );
TurnoverWindow  = Param( "Turnover averaging window (bars)", 50, 5, 250, 5 );
MinPrice        = Param( "Price floor (per share)", 5, 0, 500, 0.5 );
BenchmarkTicker = ParamStr( "Benchmark symbol", "^GSPC" );
RegimePeriod    = Param( "Benchmark trend period (bars)", 200, 20, 500, 10 );
TrendPeriod     = Param( "Instrument trend period (bars)", 200, 20, 500, 10 );
BreakoutPeriod  = Param( "Breakout lookback (bars)", 50, 5, 250, 5 );
AtrPeriod       = Param( "ATR period", 20, 2, 200, 1 );
StopAtrMult     = Param( "Stop distance (ATR multiples)", 2.5, 0.5, 10, 0.25 );

// A version string that travels with every row. When you change a rule, change
// this, and six months later the journal can still tell you which version of
// the system produced a given decision. Without it, a rule change quietly
// rewrites the meaning of every earlier row.
RuleVersion = ParamStr( "Rule set version", "v1.0" );

// ---- The same rules the candidate sheet applies ---------------------------

Turnover     = Close * Volume;
AvgTurnover  = MA( Turnover, TurnoverWindow );
SafeTurnover = Max( Nz( AvgTurnover ), 1 );
InUniverse   = Nz( AvgTurnover ) >= MinTurnover AND Close >= MinPrice;

BenchClose = Foreign( BenchmarkTicker, "C" );
BenchTrend = MA( BenchClose, RegimePeriod );
RegimeOn   = NOT IsNull( BenchClose ) AND BenchClose > BenchTrend;

Setup     = Close > MA( Close, TrendPeriod );
PriorHigh = HHV( Ref( High, -1 ), BreakoutPeriod );
Trigger   = Cross( Close, PriorHigh );

IsCandidate = InUniverse AND RegimeOn AND Setup AND Trigger;

// ---- The plan, in numbers -------------------------------------------------

Volatility   = ATR( AtrPeriod );
StopDistance = StopAtrMult * Volatility;
SafeDistance = Max( Nz( StopDistance ), 0.000001 );
StopLevel    = Close - StopDistance;

RiskBudget     = AccountEquity * RiskPercent / 100;
RiskShares     = floor( RiskBudget / SafeDistance );
ValueCapShares = floor( ( AccountEquity * MaxPositionPct / 100 ) / Max( Close, 0.000001 ) );
PlannedShares  = Min( RiskShares, ValueCapShares );
PlannedValue   = PlannedShares * Close;
PlannedRisk    = PlannedShares * StopDistance;

// ---- Output ---------------------------------------------------------------

SetOption( "NoDefaultColumns", True );

Filter = IsCandidate AND Status( "lastbarinrange" );

// Recorded before the outcome is known.
AddColumn( DateTime(), "SignalDate", formatDateTimeISO );
AddTextColumn( Name(), "Ticker", 12 );
AddTextColumn( RuleVersion, "RuleVersion", 10 );
AddColumn( Close, "SignalClose", 1.4 );
AddColumn( StopLevel, "PlannedStop", 1.4 );
AddColumn( StopDistance, "StopDistance", 1.4 );
AddColumn( PlannedShares, "PlannedShares", 1.0 );
AddColumn( PlannedValue, "PlannedValue", 1.0 );
AddColumn( PlannedRisk, "PlannedRisk", 1.0 );
AddColumn( 100 * PlannedValue / SafeTurnover, "ParticipationPct", 1.2 );
AddTextColumn( "next open", "EntryRule", 12 );

// Reserved for the human. Passing Null produces an empty column, which is the
// documented way to lay out a column the formula does not fill. These are the
// fields that turn a list of signals into a record of decisions, and they are
// the only place in the whole pipeline where "why" is allowed to be prose.
AddColumn( Null, "Taken" );
AddColumn( Null, "ActualEntry" );
AddColumn( Null, "ActualShares" );
AddColumn( Null, "ActualStop" );
AddColumn( Null, "DeviationReason" );
AddColumn( Null, "ReviewerNote" );

// With the default columns switched off, column 1 is SignalDate and column 2
// is Ticker. Sorting by ticker keeps the export stable from evening to
// evening, which matters because a diff between two journals is only readable
// if the rows are in a predictable order.
SetSortColumns( 2 );

_SECTION_END();

Download decision-journal-export.afl119 lines

How it works

The rules are a copy of the candidate sheet’s rules, which is deliberate and is also the formula’s main hazard: if the two drift apart, the journal records a plan you never saw. The comment says so, and the RuleVersion parameter is what makes a drift detectable afterwards.

SetOption( "NoDefaultColumns", True ) removes the automatic Ticker and Date/Time columns so that an export lines up with a spreadsheet’s columns. Because the ticker is genuinely needed, it is emitted explicitly as a text column in a known position. Turning the default columns off renumbers everything, which is why SetSortColumns( 2 ) sorts by ticker here and would have meant something else with the defaults left on.

The empty columns come from AddColumn( Null, "..." ), which is the documented way to lay out a column the formula does not fill. They are the human’s half of the row.

Sorting by ticker is a small decision with a practical reason: the export writes the result list as displayed, so a stable sort means two evenings’ files can be diffed.

Key functions

• SetOption( "NoDefaultColumns", True ) — removes the two predefined columns. Remember that it renumbers every column for SetSortColumns().
• AddColumn( array, name, format ) — the format is a number in WriteVal style where the fractional part is the number of decimal places; 1.4 is four decimals and 1.0 is none. The integer part is a space-padding width, not a digit count.
• AddColumn( Null, "Name" ) — a legal, documented way to produce an empty column.
• AddTextColumn( string, name, format ) — takes a single string that cannot vary bar by bar, which is exactly right for the ticker and the rule version.
• formatDateTimeISO — the YYYY-MM-DD HH:MM:SS format constant. Prefer it to formatDateTime for anything that will be parsed, because the latter follows the machine’s Windows settings and therefore differs between machines.
• ParamStr( name, default ) — used here for the rule-set version string.

Expected result

Run the exploration, then, with the New Analysis window focused, use File → Export HTML/CSV. The File menu only grows those entries while an Analysis window is active, which is why people hunt for them and conclude they do not exist.

The resulting file has one row per candidate, with the planned fields populated, the human fields empty, and no Ticker or Date/Time columns beyond the ones the formula emitted itself. Paste it under yesterday’s rows in the journal and fill in the right-hand block after the review.

=Expected result

On an evening with no candidates the export is empty, and that empty file should still be saved. A missing file and a file with no rows look identical three months later unless you keep both, and “did the process run that evening?” is a question the daily review is supposed to answer.

Test it

1. Export twice without changing anything and compare the files byte for byte. They should be identical. If they are not, something in the formula depends on the run rather than on the data.
2. Change the sort column and export again. The row order changes, which demonstrates that the export follows the display and is the reason to fix the sort once.
3. Change RuleVersion and export. Confirm the value appears in every row. Then imagine comparing a quarter of rows that lack it — that is the failure the column prevents.
4. Open the CSV in a spreadsheet and check that SignalDate parses as a date without coercion. If it does not, formatDateTimeISO has been replaced by formatDateTime somewhere.

Common errors

• The export contains Ticker and Date/Time columns you did not ask for. SetOption was called after the columns were added, or misspelled. It takes effect for the run, not retroactively for the current line.
• SetSortColumns() sorts the wrong column. The default columns were re-enabled, or added back, and every column number shifted by two.
• Rows appear for symbols with no signal. Filter is missing the IsCandidate term, or the Range is not restricted and Status("lastbarinrange") was omitted.
• The journal and the candidate sheet disagree. The two formulas’ parameters have drifted. This is the failure the RuleVersion column detects and does not prevent; the prevention is to keep both sets of parameters in one included file, which Part 11 covers.

Extension

Add the regime state and the average turnover to the export. Neither is needed for the trade, and both are needed for the post-mortem: when the quarterly review asks whether the gap was concentrated in a particular regime or in the least liquid candidates, those two columns are the difference between an answer and a hypothesis.

Recap

Strategy performance and trader execution performance are separate quantities with separate measurements. The first belongs to the rules and is reported by AmiBroker in AmiBroker’s own vocabulary, where Annual Return %, Max. system % drawdown, CAR/MaxDD and Sharpe Ratio of trades all mean something more specific than the everyday phrases they resemble. The second belongs to you, is not reported by any software, and is computed from a journal whose every pre-decision field was written before the outcome existed — including, critically, rows for the signals you skipped.

The gap between them is measured with a shadow backtest over the same period, the same universe and the same rule version, decomposed into omission, addition, timing, size and exit deviation, and expressed in R units so it adds up. A review schedule then decides when that measurement may be acted on: process changes weekly, rule changes only quarterly, and a documented decision to change nothing whenever the evidence does not support changing something — which, in a system with honest evidence behind it, is most quarters.

That closes the loop the first lesson drew. You now have a pipeline, a specification for its two constraints, a protocol for its human stage, and a way of finding out afterwards which half of the process needs attention.

Check your understanding

Question 1. Your account is down 8% over a quarter in which the shadow backtest of the same rules, over the same period and universe, shows +2%. What have you measured?

• That the strategy has stopped working
• That the strategy did what it did, and a gap of about 10 percentage points sits in your execution - the summary tells you the size of the gap but nothing about its cause
• That the backtest is unrealistic
• That the market regime changed

Show the answer and why

Answer: That the strategy did what it did, and a gap of about 10 percentage points sits in your execution - the summary tells you the size of the gap but nothing about its cause

Level 1 of the comparison sets the scale of the question and stops there. The next step is the decomposition: how much of the 10 points is omission, how much is size, how much is additions with no signal behind them. Without the journal rows for skipped signals, that decomposition cannot be computed at all.

Question 2. Which of these is the label AmiBroker actually uses in the backtest report?

• CAGR
• Maximum Drawdown
• Annual Return %
• Expectancy ($)

Show the answer and why

Answer: Annual Return %

The report row is `Annual Return %`; `CAR` is the metric string used programmatically and inside `CAR/MaxDD`. There is no row called "Maximum Drawdown" - the drawdown rows are `Max. system drawdown` and `Max. system % drawdown`. And `Expectancy ($)` only exists as a row if you add it yourself as a custom metric; the built-in row is `Avg. Profit/Loss`.

Question 3. A journal shows a size ratio distribution with two clusters, one near 1.0 and one near 0.5. What does that indicate?

• A calculation error in the sizing formula
• That an untested discretionary overlay has been added - sizing is being varied by confidence, and its effect on results has never been measured
• That the participation cap is binding on half the trades
• Nothing; size varies naturally

Show the answer and why

Answer: That an untested discretionary overlay has been added - sizing is being varied by confidence, and its effect on results has never been measured

A binding cap produces a spread of ratios rather than a second cluster at a round fraction. Two clusters at 1.0 and 0.5 is the signature of a human decision rule - full size when confident, half otherwise - operating outside the system. It may even help, but nobody knows, because it has never been through the research loop.

Question 4. The backtest showed a Max. system % drawdown of -24% and a maximum run of eleven consecutive losers. You are 15% down after six losses in a row. Which reasons would legitimately justify changing a rule? Select all that apply.

• The drawdown and the losing run are both inside what the evidence already showed
• You discover the universe was built from current index members, so the test carried membership look-ahead
• A different breakout period would have avoided four of the six losses
• Your broker changed its commission structure, invalidating the cost assumption in the test

Show the answer and why

Answer: You discover the universe was built from current index members, so the test carried membership look-ahead, Your broker changed its commission structure, invalidating the cost assumption in the test

A discovered bias means the old results were not evidence, and a changed cost structure means the rules were correct for a world that no longer exists. Both are legitimate and neither is about recent performance. A drawdown inside the tested range is the system behaving as measured, and a parameter that would have avoided recent losses always exists - finding it costs nothing and demonstrates nothing.

Question 5. Why must the shadow backtest use the same rule version as the journal rows it is compared against?

• Because AmiBroker refuses to backtest an old formula
• Because otherwise the comparison measures the rule change rather than the execution gap, and the two are indistinguishable in the result
• Because the report settings would differ
• Because the journal cannot store more than one version

Show the answer and why

Answer: Because otherwise the comparison measures the rule change rather than the execution gap, and the two are indistinguishable in the result

The comparison is a subtraction between two quantities that must differ in exactly one respect: whether the rules were executed by the machine or by you. If the rules themselves also differ, the difference contains both effects and no arithmetic can separate them afterwards. That is the whole purpose of carrying a version string on every journal row.

Check my answersClear