Building Screening Filters That Mean Something

Anybody can write Filter = RSI() < 30;. The difficulty in screening is not expressing a condition; it is knowing what each condition is for, choosing a threshold you could justify to somebody who disagreed with you, and being able to say afterwards where your universe went.

This lesson builds a screen as six named stages. The point of the naming is not tidiness. A screen that is one enormous conjunction is a screen you cannot debug, because when it returns nothing it returns no information about which part of it was responsible.

A screening funnel

1. 1UniverseWhich instruments are even candidates: not composites, not indexes, enough history
2. 2Price floorRemoves symbols whose percentage statistics are dominated by the tick size
3. 3LiquidityRemoves what you could not trade at a size that matters to you
4. 4TrendA state that has held for weeks: the context
5. 5MomentumMeasured over a different horizon, so it is not restating the trend test
6. 6VolatilityA band, not a ceiling. Too quiet is also a rejection
7. 7SetupThe one condition that is about today rather than about context

Every stage answers a different question. A stage that removes nothing is not doing any work.

Liquidity comes first, and it is not share volume

Liquidity is the first real filter in almost every professional screen, for a reason that has nothing to do with computation and everything to do with honesty: a candidate you could not trade is not a candidate, however good the chart looks. If a screen surfaces an instrument that trades forty thousand currency units a day and your intended position is ten thousand, you are not looking at an opportunity. You are looking at a chart that will move against you the moment you touch it, and a backtest of it will be fiction. Part 1 made this argument; here it becomes a line of code.

The measure to use is turnover, not share count:

Fragment — not a complete formula

Turnover    = Close * Volume;
AvgTurnover = MA( Turnover, 50 );
Liquid      = AvgTurnover > MinTurnover;

Share volume is not comparable across instruments. Ten million shares of a twenty-cent stock is two million currency units of trading; ten thousand shares of a two-hundred-unit stock is the same figure. A screen filtered on share volume systematically over-selects cheap instruments — which are also the ones with the widest relative spreads, so the error compounds.

Averaging matters too. A single day’s turnover is dominated by whatever news happened that day. Fifty bars is a reasonable default: long enough to smooth a spike, short enough to notice an instrument that has genuinely dried up.

⌀Data quality warning

Before you trust any liquidity filter, confirm what your data provider means by “volume”. Some feeds report volume in hundreds or thousands of shares; some report currency turnover directly; some report zero for indexes and composites; some report zero for every symbol because the import format definition omitted the column. Each of these produces a screen that returns nothing, or one that returns everything, and neither says why.

Price and universe filters

A price floor is not snobbery about cheap instruments. It is a statistical necessity. Below some price, the minimum tick becomes a significant percentage of the price, and every percentage-based measure you compute — returns, ATR percentage, distance from a moving average — becomes quantised and noisy. On an instrument priced at 0.30 with a tick of 0.01, a one-tick move is 3.3%. Your volatility filter is then measuring the tick size.

Universe filters are the ones people forget, and they cause the most confusing results:

• Composite tickers. If you have used AddToComposite() — Part 16’s subject — your database contains artificial symbols, conventionally prefixed with ~. They have prices and often volumes, and they sail through price and trend tests. Exclude them by name: Exclude = StrLeft( Name(), 1 ) == "~";
• Indexes. Imported index symbols behave like instruments and are not tradeable. Either exclude them by category in the Filter Settings window, or by name pattern.
• Insufficient history. A symbol listed three months ago cannot have a two-hundred-bar average. That produces Null, and Null compared with a number is not true, so the symbol vanishes — correctly, but silently. Testing for it explicitly turns “no candidates” into “these 340 symbols could not be evaluated”, which is a diagnosis rather than a mystery.

Trend filters

A trend filter states the context you require. Three common forms, in increasing strictness:

Fragment — not a complete formula

AboveAverage = Close > MA( Close, 200 );
RisingAverage = MA( Close, 200 ) > Ref( MA( Close, 200 ), -20 );
Aligned = MA( Close, 50 ) > MA( Close, 200 );

All three are states: true over spans of bars rather than on single bars. That is what a screening context wants, and it is the opposite of what a signal wants. Part 9 made this distinction; a screen is where it becomes practical, because a screen built on events returns almost nothing while a screen built on states returns almost everything, and beginners usually discover this the hard way.

Two honest caveats. First, a trend filter based on a two-hundred-bar average is late by construction; it will not be above the average at the start of a move. That is a deliberate trade, not a defect, and you should be able to say what you are trading away. Second, every trend filter is period-dependent in a way that is easy to hide from yourself. The two-hundred-bar average is a convention, not a discovery, and a screen that only works with 200 and not with 150 or 250 is telling you something uncomfortable about itself.

Momentum filters

The mistake to avoid is a momentum filter that restates the trend filter. Close > MA(Close, 200) and ROC( Close, 200 ) > 0 are close to the same statement, and combining them feels like two pieces of confirming evidence when it is one piece counted twice.

Choose a clearly different horizon:

Fragment — not a complete formula

Momentum = ROC( Close, 60 );     // three months, against a 200-bar trend context

ROC( array, periods ) returns the percentage change over the given number of bars, which is directly comparable across instruments of different prices. That comparability is why it is a better screening variable than a points difference.

∴Research note

Whenever you combine two conditions, ask what fraction of the universe passes each one alone and what fraction passes both. If passing the first makes passing the second nearly certain, the second condition is decoration. The stage audit in the next lesson measures exactly this, and it is worth running before you add any condition to a screen you already have.

Volatility filters

Volatility filters are usually written as ceilings and usually should be bands.

Fragment — not a complete formula

AtrPct = 100 * ATR( 20 ) / Close;
Acceptable = AtrPct > 1 AND AtrPct < 8;

ATR( period ) is the average true range in the instrument’s own price units. Dividing by the close normalises it, which is what makes a five-unit instrument and a five-hundred-unit instrument comparable.

The upper bound is about risk: if the average daily range is 12% of price, any stop you can afford sits inside a single day’s noise, so your exit will be triggered by nothing. The lower bound is about relevance: an instrument that moves 0.3% a day will not travel far enough to matter within your holding period, and the costs of trading it will dominate whatever it does.

Note that AtrPct divides by Close, which is why the price floor stage belongs earlier — not because the division fails, but because on very low-priced instruments the result is dominated by tick quantisation and the filter stops measuring volatility.

Setup conditions

Every stage so far describes a context that has held for weeks. The setup is the single condition that is about today, and it is the one that determines how many candidates you get on any given evening.

Fragment — not a complete formula

AtHigh   = Close >= HHV( Close, 10 );      // at the top of its recent range
Pullback = Close < MA( Close, 20 ) AND Close > MA( Close, 200 );
Squeeze  = ATR( 20 ) < Ref( LLV( ATR( 20 ), 60 ), -1 ) * 1.1;

Keep exactly one of these. A screen with three setup conditions is a screen that returns nothing, and the reason is not that the market is empty — it is that you have multiplied three independent-ish conditions each satisfied by perhaps 10% of symbols, and 0.1% of a two-thousand-symbol universe is two.

✕Common mistake

A setup written with Cross() in a screen filtered to the last bar is a common cause of an empty list. Filter = Status("lastbarinrange") AND Cross( Close, MA( Close, 50 ) ); reports a symbol only if the crossing happened on precisely that bar. That is a legitimate thing to ask for, but it will be true for a handful of symbols on a typical day and zero on many days. If you meant “recently crossed”, say so: BarsSince( Cross( Close, MA( Close, 50 ) ) ) <= 3.

The screen as six auditable stages

Complete runnable AFL

filter-stages.afl

// filter-stages.afl
// Part 12 - Building Screening Filters That Mean Something
//
// A screen written as six named, separately testable stages, plus a column
// that names the FIRST stage each symbol failed. Reporting every symbol rather
// than only the survivors is what turns a screen into something you can debug:
// when the candidate list is empty you can see at a glance where the universe
// died.
//
// Assumptions:
//   - Daily bars, Range wide enough for a 200-bar average to warm up.
//   - Thresholds below are calibrated for a large, liquid equity market quoted
//     in a currency where a million units of daily turnover is a low bar. On a
//     smaller market every one of them needs re-deriving from the data, not
//     copying. A threshold carried across markets is the most common reason a
//     screen returns nothing at all.
//   - Every stage is an absolute test against a fixed number. That is the case
//     in which the ORDER of the stages does not change the surviving set - it
//     changes only cost, diagnosis, and which stage gets the blame. Stages 5
//     and 6 show where that stops being true.

MinPrice        = 5;
LiquidityPeriod = 50;
MinTurnover     = 1000000;
TrendPeriod     = 200;
MomentumPeriod  = 60;
MinMomentum     = 0;
AtrPeriod       = 20;
MaxAtrPct       = 8;
MinAtrPct       = 1;
PullbackPeriod  = 10;

Turnover    = Close * Volume;
AvgTurnover = MA( Turnover, LiquidityPeriod );
Trend       = MA( Close, TrendPeriod );
Momentum    = ROC( Close, MomentumPeriod );
AtrPct      = 100 * ATR( AtrPeriod ) / Close;

// Stage 1 - price floor. Cheapest test, and the one that removes the symbols
// whose percentage statistics are dominated by the tick size.
Stage1Price = Close > MinPrice;

// Stage 2 - liquidity. Placed early because everything after it is a per-bar
// indicator computed over hundreds of bars, and because a candidate you could
// not trade is not a candidate however good the chart looks.
Stage2Liquid = AvgTurnover > MinTurnover;

// Stage 3 - trend. A state, not an event: true on every bar the condition
// holds, which is what a screening context wants.
Stage3Trend = Close > Trend;

// Stage 4 - momentum, measured over a different horizon from the trend test so
// that the two stages are not restating each other.
Stage4Momentum = Momentum > MinMomentum;

// Stage 5 - volatility band. Both ends matter: too quiet and the setup never
// travels far enough to matter, too wild and any stop you can afford is inside
// the noise. AtrPct divides by Close, so stage 1 protects it from the symbols
// where that division is unstable.
Stage5Volatility = AtrPct > MinAtrPct AND AtrPct < MaxAtrPct;

// Stage 6 - the setup itself. Everything above describes a context that lasts
// for weeks; this describes today. Note it is defined relative to the recent
// range, so it only means anything for symbols that survived stages 1 to 5.
Stage6Setup = Close >= HHV( Close, PullbackPeriod );

Passed = Stage1Price AND Stage2Liquid AND Stage3Trend AND
         Stage4Momentum AND Stage5Volatility AND Stage6Setup;

// The first stage that failed. Nz() matters here: an unwarmed indicator gives
// Null, Null compared with a number gives Null, and Null in a selector would
// silently pick the wrong text. Treating "not known yet" as "failed" is a
// decision, and it is written down rather than left to chance.
FailStage = IIf( NOT Nz( Stage1Price ),      1,
            IIf( NOT Nz( Stage2Liquid ),     2,
            IIf( NOT Nz( Stage3Trend ),      3,
            IIf( NOT Nz( Stage4Momentum ),   4,
            IIf( NOT Nz( Stage5Volatility ), 5,
            IIf( NOT Nz( Stage6Setup ),      6, 0 ) ) ) ) ) );

StageList = "PASSED all six stages\n" +
            "Failed 1: price floor\n" +
            "Failed 2: liquidity\n" +
            "Failed 3: trend\n" +
            "Failed 4: momentum\n" +
            "Failed 5: volatility band\n" +
            "Failed 6: setup";

// Every symbol is reported, not only the survivors. Change this line to
// "Filter = Status( "lastbarinrange" ) AND Passed;" once the stages behave.
Filter = Status( "lastbarinrange" );

AddTextColumn( FullName(), "Name", 30 );
AddMultiTextColumn( FailStage, StageList, "Outcome", 26 );
AddColumn( Passed, "Passed", 1.0 );
AddColumn( Close, "Close", 1.2 );
AddColumn( AvgTurnover, "Turnover 50d", 1.0 );
AddColumn( 100 * ( Close - Trend ) / Trend, "% from MA200", 1.1 );
AddColumn( Momentum, "60-bar ROC %", 1.1 );
AddColumn( AtrPct, "ATR %", 1.2 );

// Column 5 is "Passed": its TOTAL is the number of survivors and its COUNT is
// the size of the universe the run actually examined.
AddSummaryRows( 1 + 16, 1.0, 5 );

SetSortColumns( -5, 4 );   // survivors first, then grouped by outcome

Download filter-stages.afl106 lines

How it works

Each stage is assigned to its own named variable and nothing else. Passed is the conjunction of all six. The interesting part is FailStage, a nested IIf() chain that reports the first stage each symbol failed, and AddMultiTextColumn which turns that number into a sentence.

Nz() appears inside every test in that chain, and it is not defensive noise. Nz( x ) converts Null — and NaN, and infinity — to zero. Without it, a symbol whose two-hundred-bar average has not warmed up produces Null in Stage3Trend, NOT Null is not a useful value, and the selector lands somewhere unintended. Converting “not known” to “failed” is a decision, and writing it down is the difference between a formula that handles missing data and one that merely appears to.

The Filter line reports every symbol rather than only survivors. That inversion is the whole idea: a screen that shows only its winners cannot tell you why it has none.

Expected result

=Expected result

One row per symbol in the universe, sorted with the survivors at the top and the rest grouped by which stage killed them. Above the first row, a TOTAL for the “Passed” column — the number of survivors — and a COUNT — the number of symbols the run actually examined.

Test it

1. Universe test. Does the COUNT summary row match the number of symbols you believe are in your universe? If not, the problem is Apply to, not the code.
2. Distribution test. Read down the Outcome column. If ninety per cent of your universe fails at one stage, that stage is doing all the work and every other stage is decoration. If a stage fails nobody, delete it or tighten it.
3. Threshold test. Halve MinTurnover and re-run. The number failing at stage 2 should drop substantially. If it does not, your turnover figures are not what you think they are — go back to the data-quality question.

Common errors

✕Common mistake

Everything fails at stage 2 and the screen is empty. A turnover threshold copied from material written about a different market. Thresholds are properties of a universe. Derive them: run the screen with Filter = Status("lastbarinrange"); and no conditions, add a turnover column, sort descending, and look at where your universe actually sits.

Nothing fails at any stage. The thresholds are inert. Compare each one against the actual distribution in your universe before adjusting it: a filter that passes 99% of symbols is costing you a line of code and buying you nothing, and it is worth deciding whether you wanted a filter there at all rather than reflexively tightening it.

The Outcome column names the wrong stage. The nested IIf() chain reports the first failure in the order you wrote it. Reordering the chain reorders the blame without changing who passes.

Filter order: what it does and does not change

This is the part most screening material gets wrong in one direction or the other, so it is worth stating precisely.

When every stage is an absolute test against a fixed number, the order does not change the surviving set. A AND B AND C is the same set as C AND B AND A. If someone tells you that reordering a conjunction of fixed thresholds produced different candidates, something else changed too.

What the order does change, even then, is real and matters:

• Cost. AmiBroker evaluates the whole formula for every symbol, so ordering does not short-circuit the way it would in a procedural language. But when you split a screen across stages — a scan pass that builds a universe, then an exploration over the survivors, as the next lesson does — the order decides how much work the expensive stage has to do.
• Diagnosis. The staged report above blames the first failing stage. Put liquidity last and every illiquid symbol will be reported as a trend failure, which is true and useless.
• Numerical safety. AtrPct divides by Close. RelVolume divides by an average volume that can be zero. Ordering the guards before the computations that need them is how you keep Null and infinity out of the results.

And the order genuinely changes the answer as soon as any stage is relative. This is the case worth remembering:

Pseudocode — not valid AFL

Screen A:  take the top 20 by momentum, then keep those that are liquid
Screen B:  keep the liquid symbols, then take the top 20 by momentum

These produce different lists, and usually very different ones, because “top 20” is defined relative to whatever set it is applied to. Screen A’s twenty strongest symbols across the whole database will be dominated by small illiquid instruments, and after the liquidity filter you may be left with three. Screen B gives you twenty tradeable names. Any percentile, any top-N selection, any z-score computed across the surviving group, and any threshold derived from the data rather than fixed in advance behaves this way.

The rule that follows: absolute filters first, relative selection last. Cross-sectional ranking is Part 13’s subject, and this is the reason it appears after this part rather than inside it.

What changed in your understanding

A screen is a funnel of stages, each answering a distinct question, and it is worth writing so that you can see the shape of the funnel rather than only its output. Liquidity is measured in currency and comes early because untradeable candidates are not candidates. Momentum should not restate the trend. Volatility filters need a floor as well as a ceiling. And filter order is irrelevant to the surviving set only while every filter is absolute — the moment ranking enters, order is the whole answer.

Check your understanding

Question 1. Two screens use identical conditions: liquidity, trend, momentum, and "top 20 by momentum". Screen A ranks first then filters; Screen B filters then ranks. What should you expect?

• Identical results, since AND is commutative
• Different results, because a top-N selection is defined relative to the set it is applied to
• Screen A will be faster but otherwise identical
• Screen B will return more than 20 rows

Show the answer and why

Answer: Different results, because a top-N selection is defined relative to the set it is applied to

Absolute thresholds commute; relative selections do not. Ranking the whole database first and then filtering leaves you with however many of those twenty happened to survive — often very few, and biased towards small illiquid names.

Question 2. Why is Close * Volume preferred over Volume in a liquidity filter?

• It is faster to compute
• It produces a figure comparable across instruments of different prices
• Volume is unreliable in most databases
• It compensates for missing bars

Show the answer and why

Answer: It produces a figure comparable across instruments of different prices

Share counts are not comparable: ten million shares at 0.20 and ten thousand shares at 200 represent the same turnover. Filtering on share volume systematically over-selects cheap instruments, which are also the ones with the widest relative spreads.

Question 3. A screen returns nothing. The stage report shows 1,850 of 1,900 symbols failing at the liquidity stage. What is the most likely cause?

• The trend filter is too strict
• The threshold was calibrated for a different market, or the volume field means something other than shares
• The formula needs SetBarsRequired()
• Apply to is set to Current symbol

Show the answer and why

Answer: The threshold was calibrated for a different market, or the volume field means something other than shares

The stage report has already localised the fault. A liquidity threshold is a property of a universe and a currency, not a constant; the other possibility is that the data provider reports volume in a unit you did not expect. Apply to = Current symbol would have shown one row, not 1,900.

Question 4. Which of these is a state rather than an event, and therefore suitable as a trend filter? Select all that apply.

• Close > MA( Close, 200 )
• Cross( Close, MA( Close, 200 ) )
• MA( Close, 50 ) > MA( Close, 200 )
• BarsSince( Cross( Close, MA( Close, 50 ) ) ) <= 3

Show the answer and why

Answer: Close > MA( Close, 200 ), MA( Close, 50 ) > MA( Close, 200 )

The first and third are true over spans of bars. Cross() is true on exactly one bar, which is why using it as a screening context returns almost nothing. The fourth is a deliberate conversion of an event back into a short-lived state — useful, but a setup rather than a context.

Check my answersClear