From Hypothesis to Rules

Here is the test that decides whether your rules are finished. Hand the written page to somebody who can write AFL, who has never heard your idea, and who is not allowed to ask you a single question. Have them implement it on the same data. Then line their signals up against yours, bar by bar.

If the two sets of signals are identical, you have rules. If they differ anywhere, you do not — you have a description of a habit, and the difference marks the exact place where your own implementation has been making a decision you never wrote down. That is the ambiguity test, and this lesson is mostly about passing it.

The AmiBroker User’s Guide puts the same requirement in one sentence at the start of its backtesting chapter: you need objective, mechanical rules to enter and exit the market, and you have to work them out yourself, because they must match your own circumstances. This lesson is about what “objective” costs you in practice.

The specification sheet

A complete rule set has six sections. Anything you leave out will be filled in by an accident — a default in a settings dialog, an assumption in your formula, or whatever you happened to believe on the day you wrote it.

Section	What it fixes
Universe	Which instruments are eligible, and when
Data	Source, interval, adjustment, and what happens to missing bars
Entry	The condition, evaluated on a named bar, using only data available then
Exit	Every way a position can end — rule, stop, time, and the priority between them
Sizing	How much capital goes into one position and how many run at once
Execution	Which price, which bar, and what it costs (the next two lessons)

This lesson covers the first four. Sizing and execution get their own treatment, because they change results more than entries do and because most people set them by accident.

Making an entry objective

Take a sentence of the kind that appears in every trading book: buy strong stocks pulling back in an uptrend. Four words in it are doing work, and none of them is defined.

Word	The decisions hiding inside it
strong	Strong against what — its own past, a sector, an index? Measured over what window? Ranked, or thresholded?
pulling back	How far down, from what reference? Over how many bars? Does an intraday low count, or only a close?
uptrend	Defined by a moving average, by higher lows, by a slope, by a lookback return? Over what period? Evaluated on which bar?
buy	On the bar the condition became true, or on the next one? At what price? What if it is already held?

A defensible rewrite makes each one arithmetic, and it will be longer and less inspiring:

Entry. On the close of bar t, enter long if all of the following are true, using only data up to and including bar t: the close is above its own 200-bar simple moving average; the 20-bar return, close to close, is in the top quintile of the eligible universe on that bar; the close is at least 5% below the highest close of the prior 20 bars; and there is no position already open in this symbol.

Every clause names its data, its window and its bar. The last clause looks pedantic and is not: without it, a condition that stays true for eleven bars is eleven entry signals, and what the backtester does with the extra ten depends on a mode setting most readers have never opened. AmiBroker’s default backtest mode removes those redundant entries in the same way ExRem() would, which is convenient — but “the engine deleted them for me” is not the same sentence as “my rules say one position at a time”, and only one of those two sentences is reproducible by a second implementer.

✕Common mistake

The commonest single ambiguity in a written rule set is state versus event. “The close is above the 50-day average” is true on every bar of an advance. “The close crossed above the 50-day average” is true on one bar. Cross() returns 1 only on the bar the relationship changed. Two implementers reading the word “above” will produce entirely different trade lists, and both will be reading your sentence correctly.

One sentence, two implementations

Both rows are honest readings of “buy when the close is above the average”. Row A fires four times, row B twice, and they enter on different bars. Nothing in the English sentence chooses between them.

Bar	1	2	3	4	5	6
Close	48.0	49.5	51.0	52.0	50.5	51.5
MA(Close, 3)	Null	Null	49.5	50.8	51.2	51.3
A: Close > MA (state)	Null	Null	1	1	0	1
B: Cross(Close, MA) (event)	Null	Null	1	0	0	1

Both rows are honest readings of “buy when the close is above the average”. Row A fires four times, row B twice, and they enter on different bars. Nothing in the English sentence chooses between them. Prices in this diagram are invented for the illustration. They are not market data and nothing should be inferred from them.

The gaps that generate almost all disagreements

Work down this list for every rule you write. Each line is a place where two competent implementers routinely diverge.

• Which price? Close, open, high, low, an average, or an intraday level.
• Which bar? The bar the condition became true, the next one, or the one after the confirmation you have not defined.
• State or event? As above. Say which, in the sentence.
• Strict or inclusive? Is “above” > or >=? On equal values, real data produces ties more often than you expect, especially in low-priced instruments.
• Warm-up. A 200-bar average has no value on bar 40. Does the rule wait, or does it treat the empty value as false? Symbols with short histories are where this bites.
• Ties in a ranking. Two symbols with an identical score and one free slot. Something will break the tie; decide what.
• Already in a position? Add, replace, ignore, or reverse.
• Simultaneous entry and exit on the same bar for the same symbol. Which wins?
• Missing bars. A halted or untraded day. Does your data source leave a hole, or repeat the previous close as though trading had occurred?
• Corporate actions. An unadjusted split is a 50% gap to your rule.

Exits, and why every system needs three of them

An entry rule with no exit is not a system; it is a shopping list. Exits come in three families, and a complete specification names all three or explicitly says a family is unused.

Condition exits mirror the entry: something became true, so leave. The reverse cross, a stop-loss level being touched, a signal from an indicator. These express the reason you entered no longer applying.

Risk exits are about the size of the loss rather than the state of the market. A maximum loss stop, a trailing stop, a volatility-scaled stop. AmiBroker implements these through ApplyStop(), and Part 28 covers its full signature, its three-valued ExitAtStop parameter and the intrabar assumptions that make stops flatter results. This lesson only asks you to state whether you have one, because “no stop” is a decision with consequences and it must appear on the page as a decision rather than as an absence.

Time exits end the trade on the clock. They are the family most often left out and the one that does the most for the interpretability of a test.

Time-based exits earn their place

A holding-period cap does four useful things:

1. It bounds the question. “Does this condition predict the next 20 bars?” is a claim you can evaluate. “Does this condition predict the future?” is not.
2. It bounds the exposure. Without a time exit, a condition exit that never fires leaves capital committed indefinitely, and your Exposure % — AmiBroker’s bar-by-bar measure of how much of the portfolio was invested — stops meaning what you think.
3. It makes trades comparable. A sample of trades held between two and eleven bars tells a different story from one where a single trade ran for four years and produced most of the profit.
4. It exposes weak edges. Many effects are real for a few days and gone afterwards. Capping the hold reveals the horizon over which the effect actually lives.

In AFL you have two ways to express it. BarsSince() counts bars since an array was last true, which works on your own signal arrays. The backtester also offers an N-bar stop through ApplyStop( stopTypeNBar, stopModeBars, n ), counted from the trade entry inside the engine. The second is usually the safer choice in a system that uses trade delays, because the engine counts from the bar the trade actually happened on rather than from the bar your signal fired. The project at the end of this part uses it, and Part 28 covers the mechanics.

Whichever you use, state the priority. If the condition exit and the time exit fall on the same bar, one of them is recorded as the reason, and reports later group trades by exit reason. Decide which, and write it down.

!Important

“Whichever comes first” is a complete exit specification. “Sell on the reverse cross” is not, because it does not say what happens if the reverse cross never comes. Every rule set needs a terminal exit — a condition that must eventually be met — or the end of your test range silently becomes your exit rule, and it closes every open trade at whatever price the last bar happened to print.

Defining the universe

The universe is part of the answer. A result obtained on the largest 100 shares in a market says nothing about the smallest 1,000, and vice versa. Specify it with the same care as the entry rule:

• Membership. Which symbols, and — this is the hard part — when. A watch list you build today contains the companies that still exist and still qualify today. Testing it over fifteen years asks a question about survivors. Point-in-time membership is the fix and it is often unavailable; Part 30 covers what to do when it is.
• Liquidity floor. A minimum on average turnover, in money rather than shares, measured over a stated window and evaluated on the signal bar. Turnover, not volume: a share at 3.00 trading two million shares and one at 300.00 trading twenty thousand have the same turnover and wildly different share counts.
• Price floor. Very low-priced instruments have wide relative spreads and tick-size effects that make percentage returns misleading.
• Instrument type and currency. Ordinary shares only, or also trusts, ETFs, ADRs? One currency, or several? Mixing currencies without saying so puts an unmodelled FX return inside your result.
• Exclusions. Anything you deliberately leave out, and why. This line exists so that a reader can see the exclusions rather than infer them.

Every one of these is a filter that could have been chosen to flatter the result. Recording the choice, with the date it was made and whether it was made before or after you saw a number, is what separates a universe definition from a selection.

The ambiguity test in practice

The paired version needs a second person, and it is worth arranging once for a system you care about. Give them the written rules and nothing else. Compare signal arrays, not summary statistics: two implementations can produce similar-looking reports from very different trades, and the report will hide the disagreement.

Most of the time you will be working alone, so use the solo version:

1. Write the rules on one page. Close the formula.
2. Wait long enough that you are reconstructing rather than remembering — a few days is usually enough.
3. Implement them again from the page, in a new file, without opening the old one.
4. Compare the two signal arrays bar by bar on one symbol with a long history. An Exploration listing the bars where the two disagree takes about five minutes to write.
5. Every disagreement is an ambiguity in the page, not a bug in either formula. Fix the page.

The last step is the one that gets skipped. The temptation is to fix the newer formula to match the older one and move on. That leaves the ambiguity intact, and it will produce a third answer the next time.

▷Exercise

Try the solo test on the smallest rule you have: a single moving-average cross. Write it in English on one line. Most readers find at least two disagreements — the state-versus-event question and the warm-up question — in a rule they would have described as unambiguous.

Documenting every choice you made

By the time the rules are written, you will have made between fifteen and forty decisions. Almost all of them had a defensible alternative. The record of those decisions is a research artefact in its own right, and it matters for one specific reason: choices made before you saw a result and choices made after are different species, and only the log can tell them apart.

A workable decision log is a table with four columns:

Decision	Chosen	Alternatives considered	When
Trend definition	200-bar simple moving average	100-bar; 50/200 pair; 12-month return	Before first run
Entry timing	Event (cross), not state	State (close above average)	Before first run
Hold cap	60 bars	20; 120; none	Before first run
Liquidity floor	50-day turnover ≥ 5,000,000	20-day; 3,000,000; volume instead of turnover	Before first run
Tie-break for ranking	Most liquid candidate preferred	Random; alphabetical; highest score	After first run, when ties appeared

That last row is honest and it costs something: a choice made after seeing results is a degree of freedom you have spent, and it should be reported alongside any conclusion you draw. Part 30 shows how quickly those add up, and Part 32 shows the only reliable remedy — data the choices never touched.

∴Research note

Keep the log in the same file as the formula, or next to it in version control. A decision log in a separate document you do not open is a decision log that stops matching the code within a fortnight, and a log that disagrees with the code is worse than none.

Recap

Rules are finished when two competent implementers working independently from your written page produce identical signals. Getting there means converting every adjective into arithmetic, naming the bar each condition is evaluated on, and answering the specific questions — state or event, strict or inclusive, warm-up, ties, already-held, same-bar conflicts, missing data — that generate nearly all disagreements.

A complete exit specification names condition exits, risk exits and time exits, says explicitly when a family is unused, and includes at least one terminal exit so the end of your test range does not become an exit rule by default. Time caps deserve more use than they get: they bound the claim, bound the exposure, make the trade sample comparable and reveal the horizon over which an effect actually lives.

The universe is part of the answer, not a detail of the setup, and every filter in it is a choice that could have been made to flatter a result. The decision log records those choices and, crucially, whether each was made before or after you saw a number.

Next: turning the written rules into AFL, where the four signal variables meet the trade delay that separates knowing something from being able to act on it.

Check your understanding

Question 1. Your rules say "buy when the close is above the 50-day moving average". A second implementer produces four times as many entry signals as you do. What is the most likely cause?

Buy = Close > MA( Close, 50 );   // theirs
Buy = Cross( Close, MA( Close, 50 ) );   // yours

• They used a different data source
• They read "above" as a state and you implemented it as an event
• They forgot to set the trade delay
• Their moving average has a different warm-up period

Show the answer and why

Answer: They read "above" as a state and you implemented it as an event

A state condition is true on every bar of an advance; Cross() is true only on the bar the relationship changed. Both are correct readings of the English. The rules were the problem, not either implementation — which is exactly what the ambiguity test is designed to reveal.

Question 2. Which of these makes a time-based exit worth adding? Select all that apply.

• It bounds the horizon the claim is about, so the result is interpretable
• It bounds the size of the loss on any single trade
• It stops one very long trade from dominating the trade sample
• It keeps Exposure % meaningful by preventing capital sitting in a position indefinitely

Show the answer and why

Answer: It bounds the horizon the claim is about, so the result is interpretable, It stops one very long trade from dominating the trade sample, It keeps Exposure % meaningful by preventing capital sitting in a position indefinitely

A time cap changes what question the test answers and what the trade sample looks like. It does not bound the loss on a trade: a position can fall a long way inside the holding window, and closing on a clock says nothing about how far. That is what a risk exit is for.

Question 3. Why does the decision log record whether each choice was made before or after the first result was seen?

• Because AmiBroker stores the settings with the report anyway
• Because a choice made after seeing a result is a degree of freedom spent on the data, which weakens the evidence
• Because later choices are more likely to contain typing errors
• Because the backtester applies later settings differently

Show the answer and why

Answer: Because a choice made after seeing a result is a degree of freedom spent on the data, which weakens the evidence

Choices made in response to a result are fitted to that data by construction, however reasonable each one seems. They inflate the apparent quality of the system by an amount you cannot estimate by eye. The log makes the count visible, and the count is what determines how much fresh, untouched data you need before believing anything.

Question 4. A rule set specifies "sell on the reverse cross" and nothing else. What happens to positions where the reverse cross never occurs before the end of the test range?

• They are discarded and do not appear in the results
• They remain open, and the end of the range effectively becomes the exit rule
• AmiBroker automatically applies a 20-bar time stop
• They are counted as losing trades

Show the answer and why

Answer: They remain open, and the end of the range effectively becomes the exit rule

Open positions are still open when the data runs out, and the portfolio backtester includes them, closing them out at the close price of the last bar. Your test range has become part of your exit logic without your having chosen it. A terminal exit — a time cap or a stop — removes that dependency.

Check my answersClear