Performance: QuickAFL, Multi-Threading and Cost

Your formula does not necessarily receive the bars you think it receives. AmiBroker has a speed optimisation called QuickAFL that hands a formula only the part of the array it needs for what is currently visible or currently in range, plus a margin. Most of the time the margin is right and you never notice. AmiBroker’s own knowledge base lists four categories of code where it may not be, and in those cases the formula produces a different answer with no error, no warning and no visible difference in the output. This lesson is about recognising whether you are in one of those categories, what to do if you are, and how to measure whether any of it matters for your code.

What QuickAFL is

The idea is old and simple. A chart shows perhaps 300 bars. Computing an indicator over twenty years of history to draw 300 of them is waste. So since 2003, when AmiBroker draws an indicator it calculates only the visible portion — plus enough earlier bars that the visible portion comes out correct.

The same reasoning was extended to the Analysis window in version 5.14. There it engages only when the selected range is less than “All quotations”, and it applies to backtests, optimizations, explorations and scans alike. In the current New Analysis window this is a setting; in the older Automatic Analysis it was a checkbox marked “Use QuickAFL”.

✱AmiBroker Professional required

QuickAFL is available in both editions — it is not a Professional feature. The similarly-named QuickGFX, the experimental chart rendering engine under Tools → Preferences → Miscellaneous, is Professional-only. They are different things and it is easy to conflate them.

How the margin is worked out

This is the part worth understanding properly, because it is what tells you whether your code is safe.

AmiBroker maintains two numbers for each formula: BackwardRef, how many bars before the first needed bar it must also compute, and ForwardRef, how many after the last. It starts with BackwardRef = 30 and ForwardRef = 0 — a safety margin intended to cover simple loops and scripts — and then every built-in function call adds its own requirement.

Where the extra bars come from

1. 1Start: BackwardRef = 30, ForwardRef = 0AmiBroker’s standing safety margin for loops and scripts.
2. 2MA( C, 40 ) adds 40 past barsBackwardRef becomes 70.
3. 3Ref( ..., -1 ) adds 1 more past barBackwardRef becomes 71.
4. 4Result for Buy = C > Ref( MA(C,40), -1 )71 past bars, 0 future bars.
5. 5Ref( MA(C,50), 1 ) instead80 past bars and 1 FUTURE bar — the estimate tracks forward references too.

The worked arithmetic is AmiBroker's own, from the QuickAFL knowledge base article.

You do not have to guess at these numbers. The AFL Formula Editor’s Tools → Code check & Profile reports the required past and future bars for the formula in front of you, and charts display them when “Display chart timing” is switched on in preferences.

The four cases where results can differ

The estimate is computed from AmiBroker’s own built-in functions. It cannot be computed for code whose requirements AmiBroker cannot see. The official list of cases where QuickAFL may not give identical results is exactly four:

1. JScript or VBScript scripting inside the formula.
2. for, while or do loops that reference more than 30 past bars. The 30 is the standing margin — a loop reaching back 200 bars is outside it.
3. Any function from an external indicator DLL.
4. Recursively calculated functions — the article names very long exponential averages, and TimeFrame functions using intervals much higher than the base interval.

⚠Warning

Category 4 is the one that catches ordinary formulas written by careful people. A recursive average has no fixed lookback: every bar’s value depends on the previous bar’s, all the way back to whatever bar the formula was handed first. Start it 300 bars ago instead of 3,000 bars ago and it is a genuinely different calculation. On a chart the two versions may look identical; the numbers are not.

Two further consequences follow directly.

Close[0] is not the first bar of the database under QuickAFL. It is the first bar of the array currently in use. Any code that treats element zero as “the beginning of history” is wrong whenever QuickAFL is active — which is one more reason for the rule from Part 8 that a formula must run correctly with BarCount as small as one.

And things changed under existing formulas in version 5.30. Cum() no longer forces all bars, so a legacy formula that relied on Cum() to defeat QuickAFL now gives different results unless SetBarsRequired( sbrAll ) is added. BarIndex() also changed: since 5.30 it “returns values always starting from zero (even if QuickAFL is turned on)”, which makes the offset workaround you may still find in old forum posts unnecessary and wrong.

SetBarsRequired, and why placement matters

Fragment — not a complete formula

SetBarsRequired( backwardref = -1, forwardref = -1 );

-1 means “no change”. sbrAll, which is -2, means “use all available bars”. The official example is SetBarsRequired( -2, -2 );, commented “require ALL past and future bars — this turns OFF quickAFL”.

Where you put the call changes what it does:

• At the top, it sets the initial requirement, which every built-in function call then adds to. SetBarsRequired( 1000, 0 ); at the top means “at least 1000 past bars, plus whatever my functions need”.
• At the bottom, it overwrites the accumulated estimate, because “any call to SetBarsRequired effectively overwrites previously calculated estimate”. This is how you reduce a requirement — for instance to stop a Cum() call forcing more bars than you actually need.

The reference is equally clear about when you do not need it: “If your formula is pure AFL you don’t need to use this function at all”, and “in most cases it is not necessary (even if you are using script or DLL) because AmiBroker always provides at least 30 past data bars more than needed.”

Two related traps involving other functions:

AddToComposite internally calls SetBarsRequired( sbrAll, sbrAll ), so it switches QuickAFL off for that formula. A visible side effect is that Code check & Profile then warns about future-bar references, which is a false alarm.

StaticVarSet and StaticVarAdd do not raise the requirement. Static arrays store only the bars currently in use. The 7.00 release notes say so directly: “Be careful when using quickafl, as StaticVarAdd would not increase ‘required bars’ (as ATC does); so if you want to actually add all bars and quick afl is turned on in analysis, it is better to add SetBarsRequired(sbrAll, sbrAll).”

A probe you can run on your own data

Goal

Stop reasoning about this in the abstract. Put a formula on a chart that tells you how many bars it actually received, which dates it spans, and what a recursive average evaluates to at the last bar — then change one setting and watch what moves.

The complete formula

Complete runnable AFL

quickafl-probe.afl

// quickafl-probe.afl
// Part 36 - Performance: QuickAFL, Multi-Threading and Cost
//
// Shows, on your own database, how many bars a formula actually received and
// whether a recursively calculated average changes when QuickAFL is switched
// off.
//
// HOW TO USE IT
//   1. Apply to a chart pane and note the four numbers in the title.
//   2. Zoom in and out. BarCount moves; the AMA value may move with it.
//   3. Right-click -> Parameters, set "Force all bars" to Yes, and compare
//      the AMA value at the same zoom level.
//
// ASSUMPTIONS
//   - Daily end-of-day bars. No feed and no Professional edition required.
//   - AmiBroker 7.00.1. The sbrAll constant needs 5.20 or later; on older
//     builds the numeric literal 1000000 was used instead.
//   - Every number this prints describes YOUR database at YOUR zoom level.
//     None of it is a result to quote anywhere else.

_SECTION_BEGIN("QuickAFL Probe");

ForceAllBars = ParamToggle( "Force all bars", "No|Yes", 0 );
SmoothFactor = Param( "AMA smoothing factor", 0.01, 0.001, 0.2, 0.001 );

// Placement matters. At the TOP of a formula SetBarsRequired sets the starting
// requirement that every built-in function call then adds to. sbrAll (-2) for
// both arguments means "all past and future bars", which turns QuickAFL off
// for this formula.
if( ForceAllBars )
    SetBarsRequired( sbrAll, sbrAll );

// AMA is recursive: every bar's value depends on the previous bar's value, all
// the way back to the first bar the formula was handed. Long recursive
// averages are on AmiBroker's own list of cases where the QuickAFL estimate of
// how many extra bars are needed may not be enough.
Slow = AMA( Close, SmoothFactor );

LastBar = BarCount - 1;

// Assign before subscripting. AFL is much happier indexing a named array than
// the result of a call, and the named version is what the Watch window can see.
Stamps = DateTime();

Plot( Close, "Close", colorDefault, styleCandle );
Plot( Slow, "AMA(" + NumToStr( SmoothFactor, 1.3 ) + ")",
      colorBlue, styleLine | styleThick );

Title =
    "Bars in this run (BarCount): " + NumToStr( BarCount, 1.0 ) + "\n" +
    "First bar in the array:      " + DateTimeToStr( Stamps[ 0 ] ) + "\n" +
    "Last bar in the array:       " + DateTimeToStr( Stamps[ LastBar ] ) + "\n" +
    "AMA at the last bar:         " + NumToStr( Slow[ LastBar ], 1.6 ) + "\n" +
    "Force all bars:              " + WriteIf( ForceAllBars, "Yes", "No" );

_SECTION_END();

Download quickafl-probe.afl56 lines

How it works

ParamToggle provides a switch. When it is on, SetBarsRequired( sbrAll, sbrAll ) runs at the top of the formula and QuickAFL is off for this execution. AMA( Close, SmoothFactor ) with a small factor is a deliberately long recursive average — category 4 above. The title reports BarCount, the first and last timestamps in the array, and the AMA value at the final bar.

Key functions

AMA( ARRAY, SMOOTHINGFACTOR ) is the adaptive/exponential average; its recursion is what makes it sensitive to where the array starts.

DateTime() returns the bar timestamps as an array; DateTimeToStr( NUMBER, mode = 0 ) formats one of them. Note the formula assigns Stamps = DateTime(); before subscripting — naming an array before indexing it is both safer and, as the next lesson shows, what makes it visible to the debugger’s Watch window.

Expected result

With “Force all bars” set to No and the chart zoomed to a few hundred bars, BarCount will be far smaller than your database and the first-bar date will be recent. Switch to Yes and BarCount should jump to the full history, with the first-bar date moving back accordingly.

Whether the AMA value changes between the two settings depends on your data, your smoothing factor and your zoom. That is the point of running it rather than being told a number.

Test it

1. Fix the zoom. Toggle “Force all bars” and record the AMA figure both ways. If they differ, you have reproduced category 4 on your own machine.
2. Leave “Force all bars” on and change the zoom. The AMA value should now be stable, because the calculation no longer depends on what is visible.
3. Set the smoothing factor to something large, say 0.2, and repeat step 1. The dependence should shrink or vanish, because a fast average forgets its starting point quickly.

Common errors

✕Common mistake

Concluding “QuickAFL is broken” and putting SetBarsRequired( sbrAll, sbrAll ) at the top of every formula. That switches off an optimisation that exists for good reason, and on a large database it makes charts noticeably slower to redraw. Use it where a documented category applies, and leave it off elsewhere.

Putting SetBarsRequired at the bottom of the formula when you meant to raise the requirement. At the bottom it overwrites the accumulated estimate, so it can silently lower it instead.

Comparing the AMA value across two different zoom levels and two different toggle settings at once, and not knowing which change caused what. Change one thing at a time.

Extension

Add a second average with a much larger smoothing factor and display both. Then add Cum( 1 ) to the formula and watch whether the reported BarCount moves — it should not, in 5.30 and later, which is the behaviour change described above made visible.

Multi-threading, and its limits

The model is one sentence: one operation on one symbol is one thread. From that everything else follows. Three chart panes on one symbol use three threads. One exploration over 400 symbols is 400 units of work. P operations over N symbols is P × N.

Where the parallelism is, and where it is not

a portfolio backtest, top to bottom

1. Phase I — run your formula on every symbolMulti-threaded. Roughly 95% of the time in a multi-symbol portfolio test.
2. Phase II — signal processing, position sizing, reportSingle-threaded. It talks to the UI and to COM, so it cannot be parallelised.
3. Custom backtester procedureRuns inside Phase II, so it is single-threaded too.

The ceiling is set by three things.

Your edition. Standard is limited to 2 threads per Analysis window; Professional to 32. Both are further capped by the number of logical processors Windows reports. Charts are multi-threaded in both editions.

Amdahl’s law. If 95% of the work is parallel, the best possible speed-up is twentyfold no matter how many cores you buy. In a multi-symbol portfolio backtest Phase I is around that 95%, so scaling is good. A single-symbol backtest is one thread, because an individual backtest is a portfolio backtest on one symbol — and even run over a watch list it proceeds sequentially. Version 5.70 added multi-threaded individual optimization, but only for exhaustive optimization and not with a custom backtester.

What your formula does. Three specific constructs give the parallelism away:

• OLE / CreateObject. Every OLE call is serviced by the single UI thread. Calls from a worker thread can be up to 30 times slower, can deadlock — AmiBroker may take up to ten seconds to break one — and reduce multiple threads to single-thread performance. This is Warning 503. The documented replacements are RequestTimedRefresh() instead of AB.RefreshAll(), SetOption( "RefreshWhenCompleted", True ) after a run, and GetOption( "FilterIncludeWatchlist" ) instead of reading the Analysis filter over OLE.
• Foreign and AddToComposite. Any access to a symbol other than the current one takes a global lock. Reduce their use; static variables are the documented alternative.
• Status("stocknum") == 0 blocks. These are meant to serialise the first symbol, which is correct and necessary — but be aware you are paying for it, and put as little inside as possible.

✕Common mistake

Foreign( "~~~Equity", "C" ) during a backtest returns the previous backtest’s equity. The New Analysis window keeps a private copy of equity during the run and only writes it to ~~~Equity when the run completes. The symptom is a system that appears to work and cannot be reproduced. To read current equity you need the custom backtester’s bo.Equity, or bo.EquityArray (5.50.1 and later).

Measuring cost

Do not tune by intuition. Four instruments, in order of usefulness:

Tools → Code check & Profile, in the Formula Editor, reports which functions are called how many times and which cost the most. The documented advice is to start with the most-called ones and ask whether they are loop-invariant.

GetPerformanceCounter( bReset = False ) returns milliseconds with resolution down to about a microsecond. Reset it at the start of the block you are timing, because the raw value is milliseconds since boot and floating-point resolution will swamp a small measurement:

Fragment — not a complete formula

GetPerformanceCounter( 1 );          // reset
med = ( H + L ) / 2;
ArrayTime = GetPerformanceCounter( 1 );   // read and reset again

Its own overhead is documented as about 0.015 ms, and it is unreliable on machines with CPU clock-stepping enabled in the BIOS.

#pragma maxthreads N limits the Analysis window to N threads. #pragma maxthreads 1 is the documented way to take a single-threaded baseline before claiming a change made things faster. Note the pragma’s own warning: between #pragma and its option there must be exactly one space.

The status bar Load Factor is AmiBroker’s own health measure: chart refresh time plus data access time plus a virtual-memory term. It reaches 100% at 200 ms chart refresh or 200 ms data access. Keep it at or below 100%; 200% is described as the practical maximum for normal operation.

Practical optimisation techniques

The guide’s own headline is unambiguous: “Poor formula coding is the foremost reason for slowdown.”

Replace loops with array expressions. The measured example is med = (H+L)/2 over 350,000 bars: 100 ms as a loop, 2 ms as an array expression — fifty times. Even over 300 bars the array version is ten times faster. Loops are documented as 10–50 times slower generally.

Move loop-invariant code out of the loop. The guide’s example is a formula that recomputes MA( C, 10 ) on every iteration. Compute it once, before the loop.

Inside a loop, use scalars or [ ] element access only. An array operation inside a loop does the whole array’s work on every iteration.

Prefer static variables to AddToComposite/Foreign for passing arrays between formulas. The documented reason is simply that they are faster — with the read-once/write-once caveat from the previous lesson.

Mind the database “Number of bars” setting. Each bar costs 40 bytes and AmiBroker allocates that many bars per symbol. 100,000 bars is 4 MB per symbol; with a 500-symbol cache that is 2 GB. The guidance is not to exceed the CPU’s on-chip cache — with a 4 MB cache, not much beyond 100,000 bars.

What changed

You now know that “how many bars does my formula get?” is a real question with a real answer, that the answer changes with zoom and range, and that four documented categories of code can give different results because of it. You know SetBarsRequired behaves differently at the top of a formula than at the bottom. You know the threading model is one operation on one symbol per thread, capped at 2 or 32 by edition, with Phase II of a backtest always single-threaded. And you have three instruments — Code check & Profile, GetPerformanceCounter and #pragma maxthreads 1 — for finding out whether any of it is actually costing you time, before you change anything.

Check your understanding

Question 1. For `Buy = C > Ref( MA( C, 40 ), -1 );` how many past bars does AmiBroker decide it needs?

• 40
• 41
• 71
• All of them

Show the answer and why

Answer: 71

The estimate starts at 30 — AmiBroker’s standing safety margin — and each built-in call adds its own requirement: 30 + 40 for the average + 1 for the Ref shift.

Question 2. Which of these is on AmiBroker’s documented list of cases where QuickAFL may give different results? Select all that apply.

• A for loop that reads 200 bars back
• A very long exponential average
• Cross( MACD(), Signal() )
• A function from an external indicator DLL

Show the answer and why

Answer: A for loop that reads 200 bars back, A very long exponential average, A function from an external indicator DLL

The four documented categories are scripting, loops reaching further back than the 30-bar margin, external DLL functions, and recursively calculated functions. Ordinary built-in calls are exactly what the estimate is computed from.

Question 3. You put `SetBarsRequired( 1000, 0 );` on the last line of a formula that uses Cum(). What happens?

• The formula gets 1000 bars plus whatever Cum() needs
• It overwrites the accumulated estimate, so the formula gets 1000 past bars
• It has no effect; only the first call in a formula counts
• It raises an error because Cum() forces all bars

Show the answer and why

Answer: It overwrites the accumulated estimate, so the formula gets 1000 past bars

Any call to SetBarsRequired overwrites the previously calculated estimate. At the top it sets a floor that functions add to; at the bottom it replaces the total — which is how you deliberately reduce a requirement.

Question 4. A portfolio backtest over 500 symbols runs on a 16-core Professional machine. Which part is single-threaded?

• Phase I, running the formula on each symbol
• Phase II — signal processing, sizing, report and any custom backtester procedure
• Neither; both phases are parallel
• Both; portfolio backtests are always single-threaded

Show the answer and why

Answer: Phase II — signal processing, sizing, report and any custom backtester procedure

Phase I parallelises across symbols and is around 95% of the work. Phase II talks to the UI and to COM, so it cannot be parallelised — which is also why a custom backtester procedure never runs in parallel.

Check my answersClear