Counting and Accumulating: BarsSince, Cum, Sum

Before a rule is worth backtesting, three questions about it are worth answering cheaply. How often does it fire? When did it last fire? Over how much of the record is it true? A rule that fires four times in twenty years cannot be evaluated statistically. A rule that fires on forty per cent of bars is a description of the market, not a filter. Both are easy to spot in about a minute, and almost nobody looks.

The three functions in this lesson answer those questions. They also supply the raw material for a large family of derived measures — momentum, participation, persistence, time in trade — and one of them carries a version-dependent behaviour change that still catches people out fifteen years later.

Three ways to count

Function	Question it answers	Shape of the answer
BarsSince( array )	How long since this was last true?	A ramp that resets at every occurrence
Sum( array, periods )	How many of the last N bars were true?	A bounded count, 0 to N
Cum( array )	How many since the first delivered bar?	A staircase that never falls

All three take arrays and return arrays. Sum is windowed; Cum is not. BarsSince measures time; the other two measure quantity.

The same condition, counted three ways

Cum accumulates without limit; Sum only sees its window. The first two Sum cells are marked unknown because the warm-up behaviour of Sum is not stated in the official documentation.

Bar	0	1	2	3	4	5	6	7
Condition	1	0	1	1	0	0	1	0
Cum( Condition )running total, never falls	1	1	2	3	3	3	4	4
Sum( Condition, 3 )window of 3, including this bar	?	?	2	2	2	1	1	1
Cum( 1 )a bar counter starting at 1	1	2	3	4	5	6	7	8

Cum accumulates without limit; Sum only sees its window. The first two Sum cells are marked unknown because the warm-up behaviour of Sum is not stated in the official documentation.

BarsSince: the age of the last event

Fragment — not a complete formula

BarsSince( ARRAY )

The documentation defines it as the number of bars that have passed since ARRAY was true, where true means 1. It converts an event into a ramp, which makes it the natural partner of Cross():

Fragment — not a complete formula

// Within five bars of the crossover, rather than only on it.
RecentCross = BarsSince( Cross( Close, MA( Close, 50 ) ) ) < 5;

That single line is the standard way to widen a one-bar event into a short window of eligibility — a setup that stays valid for a few bars while you wait for a trigger. Part 27 builds systems on exactly this shape.

The two numbers the documentation does not give you

BarsSince has two edge cases that almost every rule depends on, and the official page states neither.

The first is the value on the bar where the condition is true. Zero bars have passed, or one? The page does not say. If you write BarsSince( event ) < 5 the difference decides whether the event bar itself is included in your window.

The second is the value before the condition has ever been true. At the left edge of a chart, and on any symbol where the condition has never fired, there is no last occurrence to measure from. The page does not say what comes back.

∴Research note

Both of these are worth ten minutes of your own testing, once, on your own build. Put BarsSince( Cross( Close, MA( Close, 50 ) ) ) in an Exploration next to BarIndex() and the condition itself, run it over all quotations on one symbol, and read the first fifty rows plus any row where the condition is true. Write the two answers in a comment at the top of your personal library file. This course does not print them because they are not in the documentation, and a number repeated from memory is exactly the kind of claim that quietly becomes wrong.

The second gap has a practical consequence you can defend against without knowing the answer:

Fragment — not a complete formula

EverHappened = Cum( Condition ) > 0;
ReportedAge  = IIf( EverHappened, BarsSince( Condition ), Null );

Cum( Condition ) > 0 is a clean, documented test for “has this ever been true in the delivered range”. Guarding the age with it means a symbol where the condition never fired reports an empty cell rather than a number that might be mistaken for a real age. In a screening formula that distinction decides which symbols appear.

BarsSince converts an event into a ramp

The shape is certain: it resets at every occurrence and rises by one on every bar between. Only the starting value, and the value before the first event, are unstated.

Bar	0	1	2	3	4	5	6	7
Event	0	0	1	0	0	0	1	0
BarsSince( Event )a is the event-bar value: 0 or 1, undocumented	?	?	a	a+1	a+2	a+3	a	a+1

The shape is certain: it resets at every occurrence and rises by one on every bar between. Only the starting value, and the value before the first event, are unstated.

Sum: counting inside a window

Fragment — not a complete formula

Sum( ARRAY, periods )

A rolling sum over a fixed number of lookback periods, including today — the official page says so, in the same bracketed phrase used by HHV and LLV. Like them, it accepts a time-variant period.

Applied to a Boolean array, Sum counts:

Fragment — not a complete formula

// At least three of the last ten bars closed up.
Persistent = Sum( Close > Open, 10 ) >= 3;

// How much of the last year was spent above the 200-bar average?
YearAbove = 100 * Sum( Close > MA( Close, 200 ), 250 ) / 250;

Applied to a price array, it is an unnormalised average:

Fragment — not a complete formula

// A hand-rolled 14-bar simple moving average.
Average14 = Sum( Close, 14 ) / 14;

The window-includes-today rule has the same consequence it had for HHV. “The sum of the previous five bars, not counting this one” is not Sum( X, 5 ):

Fragment — not a complete formula

PrevFive = Sum( Ref( X, -1 ), 5 );   // the five bars before this one

✦Tip

Sum( condition, N ) is the most under-used filter in AFL. Instead of asking whether something is true now, it asks how much of the recent past it was true for. That is usually a more stable question, because a single bar’s value can flip on one tick while a count over ten bars cannot.

Like HHV and LLV, Sum has no documented warm-up behaviour for the first periods - 1 bars. The same measurement habit applies: put it in an Exploration with IsNull() beside it and look.

Cum: the running total, and a version trap

Fragment — not a complete formula

Cum( ARRAY )
Cum( Value )

A cumulative sum from the first period in the chart. A constant is broadcast to every bar, which is why Cum( 1 ) is the classic bar counter: it rises by one on every bar and starts at 1.

Cum applied to a Boolean array counts true bars: Cum( Close > Open ) is the number of up bars so far. Applied to an event array it counts occurrences, which is what makes Cum( Condition ) > 0 a reliable “has it ever happened” test.

Cum(1) and BarIndex() are not the same number

Both count bars; they start from different places.

Fragment — not a complete formula

// Documented relationship: BarIndex() == Cum(1) - 1
FirstBarByIndex = BarIndex() == 0;
FirstBarByCum   = Cum( 1 ) == 1;

BarIndex() is zero-based, Cum( 1 ) is one-based, and the BarIndex page states that BarIndex() is much faster in indicators. Prefer it, and be careful when reading older code that mixes the two — an off-by-one here is invisible on a chart and quietly wrong in a rule.

The documented last-bar idioms show the same pair:

Fragment — not a complete formula

ThisIsLastBar = BarIndex() == LastValue( BarIndex() );   // modern form
ThisIsLastBar = Cum( 1 ) == LastValue( Cum( 1 ) );       // older form

The 5.30 change

This one deserves a callout because a great deal of published AFL predates it.

!Important

Since AmiBroker 5.30, Cum() no longer forces all bars to be processed. In earlier versions a Cum() call effectively switched QuickAFL off, so cumulative statistics were computed over the entire history. The official page states the change and warns that it “may lead to different results when comparing with old versions”. To restore all-bars behaviour, add SetBarsRequired( sbrAll ); anywhere in the formula.

The practical consequence: a cumulative count in a chart pane may be counting over the trimmed range that QuickAFL delivered, not over the whole database. If your formula’s answer changes when you zoom, that is why. Any tutorial that tells you Cum() guarantees a full-history calculation was written before version 5.30 and is out of date.

Rate of occurrence, and what it is worth

Put the three functions together and you can describe a condition rather than merely detect it:

Fragment — not a complete formula

Occurrences = Cum( Condition );
BarsSoFar   = Cum( 1 );
RatePct     = 100 * Occurrences / BarsSoFar;

A rate is a genuinely useful sanity check, and it is worth being clear about what it is and is not. It is a description of this data set over this range. It is not an estimate of how often the condition will occur next year, and treating it as one is the beginning of a long line of errors that Part 30 catalogues.

What it is good for is triage. Three rough bands, offered as habits rather than rules:

• Under about one occurrence a year per symbol. There may be too few events to say anything statistically, however good the idea looks. Consider widening the universe rather than loosening the rule.
• Somewhere between a handful and a few dozen a year. Workable, and the region most tradeable setups live in.
• True on a large fraction of all bars. This is a state, not a trigger. It may be a fine market-regime filter; it is not an entry.

Measuring a condition before you trade it

Goal

Turn the three questions at the top of this lesson into one Exploration you can point at any watch list: how often does this fire, how long since it last fired, and what fraction of the record does it cover?

The complete formula

Complete runnable AFL

occurrence-statistics.afl

// occurrence-statistics.afl
// Part 9 - Counting and Accumulating: BarsSince, Cum, Sum
//
// One row per symbol. For a condition defined once at the top it reports:
//
//   how many bars since it was last true          BarsSince()
//   how many times it was true in the last N bars Sum( condition, N )
//   how many times it was true in the whole range Cum( condition )
//   the rate of occurrence, as a percentage of bars
//
// Point it at a watch list to find out which instruments a rule actually fires
// on, and how often, before writing a single line of backtest code. A rule that
// fires four times in twenty years is not a strategy; a rule that fires on
// forty per cent of all bars is not a rule.
//
// Assumptions:
//   - Status( "lastbarinrange" ) reduces the output to one row per symbol.
//     Without it the exploration prints one row per bar per symbol.
//   - Counts cover the bars DELIVERED to this run. SetBarsRequired( sbrAll )
//     asks for the full history; without it, QuickAFL may deliver fewer.
//   - A condition that has never been true has no "bars since". That case is
//     reported explicitly rather than printed as a misleading number.
//   - Occurrence counts describe the past of this data set. They are not an
//     estimate of how often the condition will occur next year.

SetBarsRequired( sbrAll );

RecentWindow = 250;   // roughly one year of daily bars

// ---------------------------------------------------------------------------
// The condition under study - replace this one line to study something else
// ---------------------------------------------------------------------------
Condition = Cross( RSI( 14 ), 70 );

// ---------------------------------------------------------------------------
// Measurements
// ---------------------------------------------------------------------------

// BarsSince turns an EVENT into a ramp: 0 or 1 on the event bar (confirm which
// on your own build), then one more on each bar that follows.
AgeBars = BarsSince( Condition );

// Cum accumulates from the first delivered bar, so Cum( condition ) is a running
// count of true bars and Cum( 1 ) is a running bar number starting at 1.
TotalCount = Cum( Condition );
BarsSoFar  = Cum( 1 );

// Sum is the windowed relative: how many of the last RecentWindow bars were true.
RecentCount = Sum( Condition, RecentWindow );

EverHappened = TotalCount > 0;
RatePct      = 100 * TotalCount / BarsSoFar;

// Before the condition has ever been true there is no meaningful age. Null is
// the honest answer, and an empty cell is easier to read than a wrong number.
ReportedAge = IIf( EverHappened, AgeBars, Null );

Filter = Status( "lastbarinrange" );

AddColumn( BarsSoFar,    "Bars in range",                                       1.0 );
AddColumn( EverHappened, "Ever occurred (1=yes)",                               1.0 );
AddColumn( ReportedAge,  "Bars since the last occurrence",                      1.0 );
AddColumn( RecentCount,  StrFormat( "Occurrences in last %g bars", RecentWindow ), 1.0 );
AddColumn( TotalCount,   "Occurrences in range",                                1.0 );
AddColumn( RatePct,      "Occurrence rate, % of bars",                          1.2 );

Download occurrence-statistics.afl65 lines

How it works

The condition lives on one clearly marked line so that the formula is reusable — change that line and everything below it re-measures the new condition.

The measurement block uses each function for what it is good at: BarsSince for the age, Cum for the range-wide totals, Sum for the recent count. The age is then wrapped in an IIf() guarded by EverHappened, so a symbol where the condition never occurred reports an empty cell rather than a number.

Filter = Status( "lastbarinrange" ) is what reduces the output to one row per symbol. Without it, an Exploration emits a row for every bar that passes the filter, which the User’s Guide points out explicitly; the "lastbarinrange" code is the documented idiom for one-row-per-symbol reports. SetBarsRequired( sbrAll ) at the top makes the cumulative columns cover the whole history rather than whatever QuickAFL felt like delivering.

Key functions

• Cum( ARRAY ) — running totals, including the bar counter Cum( 1 ).
• Sum( ARRAY, periods ) — the windowed count.
• BarsSince( ARRAY ) — the age of the last occurrence.
• Status( "lastbarinrange" ) — an array that is true on the last bar of the Analysis range. The documented way to produce one row per symbol.
• IIf( EXPRESSION, TRUE_PART, FALSE_PART ) — used here to substitute Null for a meaningless age. Remember that IIf evaluates both branches, so it guards the reporting, not the calculation.

Expected result

=Expected result

One row per symbol, with six columns. On a broad watch list you should see a wide spread in the occurrence rate — some symbols where the condition fires several times a year, some where it has not fired for years, and usually a few where it has never fired at all and the age column is empty.

Test it

Two checks are worth doing before trusting any of the numbers.

First, arithmetic: Occurrences in range divided by Bars in range, times 100, must equal the reported rate. If it does not, the columns are not measuring what their captions claim.

Second, the boundary: temporarily replace the condition with Close > 0, which is true on every bar of any sane price series. The occurrence count should then equal the bar count, the rate should read 100, and the age should be the smallest value BarsSince produces on a true bar — which is a neat way of settling the first of this lesson’s two undocumented questions.

Common errors

• Omitting Status( "lastbarinrange" ). You get one row per bar per symbol, which on a large watch list is hundreds of thousands of rows.
• Reading the age column when the condition never fired. The guard exists precisely because that number would be meaningless; if you remove the guard, do not then trust the column.
• Comparing rates between symbols with different history lengths. A symbol with three years of data and one with thirty are not directly comparable. Report the bar count alongside the rate, as this formula does.
• Treating the rate as a forecast. It describes the range you measured.

Extension

Add a column for the longest gap between occurrences, which tells you whether a rule with a respectable average rate actually goes quiet for years at a time. The running maximum of BarsSince( Condition ) gives it: Highest( BarsSince( Condition ) ). A rule that fires twenty times in ten years is a different proposition if fifteen of those firings were in one eighteen-month stretch.

SumSince: the accumulate-since-an-event idiom

One combination comes up often enough to have its own function: summing something from the moment a condition became true.

Fragment — not a complete formula

SumSince( condition, array, incFirst = False )

The official page gives its two slower equivalents explicitly — Cum( array ) - ValueWhen( condition, Cum( array ) ) and Sum( array, BarsSince( condition ) ) — and states that SumSince does the same thing much faster. The optional third argument decides whether the value on the condition bar itself is included.

That is worth knowing for two reasons. It is the right tool for “how much volume has traded since the breakout” or “how many up bars since entry”. And its documentation is a useful demonstration that BarsSince() can be used as a variable period for a windowed function, which is the same time-variant period idea that appeared with Ref() in the first lesson of this part.

Recap

Three functions, three different questions. BarsSince measures the age of the last occurrence and turns an event into a ramp — with two edge values that the documentation leaves unstated, both of which you should measure once and record. Sum counts inside a fixed window that includes the current bar. Cum accumulates from the first delivered bar and, since version 5.30, no longer forces AmiBroker to deliver all of them.

Put together they let you describe a condition before committing to it: how often, how recently, over how much of the record. That description costs a minute and regularly saves a week, because it catches rules that are too rare to evaluate and rules that are really states in disguise — which is the subject of the two lessons that follow.

Check your understanding

Question 1. What is the difference between these two lines?

A = Sum( Close > Open, 10 );
B = Cum( Close > Open );

• None - Sum with a large period is the same as Cum
• A counts up bars in the last ten bars; B counts every up bar since the first delivered bar
• A returns a number and B returns an array
• B is faster

Show the answer and why

Answer: A counts up bars in the last ten bars; B counts every up bar since the first delivered bar

Sum is windowed and bounded between 0 and 10. Cum is unbounded and rises for the whole history. Both return arrays; neither is a single number.

Question 2. Which expression is a reliable test for "this condition has occurred at least once in the delivered range"?

• BarsSince( Condition ) > 0
• Cum( Condition ) > 0
• Sum( Condition, 250 ) > 0
• NOT IsNull( BarsSince( Condition ) )

Show the answer and why

Answer: Cum( Condition ) > 0

Cum counts every occurrence from the first bar, so a positive total means at least one occurrence. The Sum version only sees the last 250 bars. The two BarsSince versions depend on values the official documentation does not state.

Question 3. A chart formula shows a cumulative count. Zooming the chart out changes the number. Why?

• A bug in the formula
• Since version 5.30, Cum() no longer forces all bars to be processed, so QuickAFL may deliver a trimmed range
• Cum() resets whenever the chart is redrawn
• The database changed

Show the answer and why

Answer: Since version 5.30, Cum() no longer forces all bars to be processed, so QuickAFL may deliver a trimmed range

The 5.30 change means Cum() no longer requests every bar. SetBarsRequired( sbrAll ) restores all-bars processing; older tutorials predating 5.30 claim Cum() does this automatically.

Question 4. You want the total volume of the five bars before this one, excluding the current bar. Which is correct?

• Sum( Volume, 5 )
• Sum( Volume, 6 ) - Volume
• Sum( Ref( Volume, -1 ), 5 )
• Both the second and the third

Show the answer and why

Answer: Both the second and the third

The documented window includes today, so Sum( Volume, 5 ) covers this bar and four others. Shifting the input back a bar, or summing six and subtracting today, both give the five previous bars.

Question 5. An exploration using Filter = Condition returned 41,000 rows for 60 symbols. What is the most likely cause?

• The condition is broken
• The range is wrong
• The filter is a state rather than an event, so it is true on many consecutive bars and each one becomes a row
• AmiBroker has a row limit

Show the answer and why

Answer: The filter is a state rather than an event, so it is true on many consecutive bars and each one becomes a row

An Exploration emits one row per bar that passes the filter. A state-shaped condition passes on long runs of bars. Status( "lastbarinrange" ) gives one row per symbol; the next lesson but one deals with the underlying state-versus-event problem.

Check my answersClear