Null Values, Nz() and Warm-Up Periods

Every array you have built in this part started with a gap. MA( Close, 3 ) had two empty values at the front. Ref( Volume, -1 ) had one. A 200-bar average has roughly two hundred. Those gaps have a name — Null — and this lesson is about what they do to everything downstream, because the answer is not what most people assume and the failure it causes produces no error message at all.

By the end you will be able to say exactly what Null means, predict where it appears, watch it travel through arithmetic and through comparisons, and write conditions that state what should happen on bars where the answer is genuinely unknown.

What Null means

Null is AmiBroker’s marker for “there is no value at this bar”. The documentation also writes it as {empty}, which is the same thing and appears in several function pages.

It is not zero. It is not false. It is not a small number. It is the absence of an answer, and the whole point of having it is that “I do not know” and “no” are different claims about the world. An indicator that has not accumulated enough data does not have a value of zero; it has no value.

The commonest source is a warm-up period: the stretch at the start of an array where a calculation does not yet have enough history behind it.

Two warm-ups, from AmiBroker's own worked table

One empty value for a one-bar shift; two for a three-bar average. Neither is a defect.

Bar	0	1	2	3	4	5	6	7	8	9
Volume	8310	3021	5325	2834	1432	5666	7847	555	6749	3456
Ref( Volume, -1 )	Null	8310	3021	5325	2834	1432	5666	7847	555	6749
Close	1.23	1.26	1.24	1.28	1.25	1.25	1.31	1.30	1.32	1.28
MA( Close, 3 )	Null	Null	1.243	1.260	1.257	1.260	1.270	1.287	1.310	1.300

One empty value for a one-bar shift; two for a three-bar average. Neither is a defect. Prices in this diagram are invented for the illustration. They are not market data and nothing should be inferred from them.

The official table shows a three-bar simple moving average beginning with two empty values, so MA( Close, n ) starts with n − 1 of them. For many other functions the guide simply does not state the warm-up length, and guessing is not good enough. NullCount() answers the question for whatever function and whatever version is in front of you:

Fragment — not a complete formula

LeadingEmpties = NullCount( MA( Close, 50 ), 1 );

The second argument selects what to count: 1 counts consecutive empty values at the beginning of the array (the default), 2 at the end, 3 at both ends, and 0 counts every empty value anywhere in the array, including isolated ones in the middle. The last of those is a data-quality tool as much as a debugging one.

⌀Data quality warning

Warm-up is not the only source. Empty values also appear in the middle of a series when data is missing — a symbol that did not trade, a vendor gap, a holiday that one exchange observed and another did not. NullCount( array, 0 ) compared against NullCount( array, 1 ) tells you whether the gaps are only at the front, where you expect them, or scattered through the history, where you did not.

Null propagates through arithmetic

Arithmetic on an unknown value yields an unknown value. Null + 1 is Null. Null * 0 is Null. Nothing rescues it, and the emptiness travels down every chain of calculations that touches it:

Fragment — not a complete formula

Average  = MA( Close, 50 );        // empty for the first 49 bars
Distance = Close - Average;        // therefore also empty for the first 49 bars
Percent  = 100 * Distance / Close; // and so is this

That much is intuitive. The next part is not.

Null propagates through comparisons too

This is the fact that catches almost everyone. A comparison against an unknown value does not return false. It returns Null.

Null travelling through a comparison and an AND

Bars 0 and 1 of Buy are not 0. They are Null — inherited from Cond1, which inherited them from the average.

Bar	0	1	2	3	4	5	6	7	8	9
MA( Close, 3 )	Null	Null	1.243	1.260	1.257	1.260	1.270	1.287	1.310	1.300
Cond1 = Close < MA( Close, 3 )	Null	Null	1	0	1	1	0	0	0	1
Cond2 = Volume > Ref( Volume, -1 )	Null	0	1	0	0	1	1	0	1	0
Buy = Cond1 AND Cond2	Null	Null	1	0	0	1	0	0	0	0

Bars 0 and 1 of Buy are not 0. They are Null — inherited from Cond1, which inherited them from the average.

Look at bar 1 of Buy. Cond2 there is a definite 0. In ordinary logic, false AND anything is false, so you might expect Buy to be 0. It is Null, because Cond1 is Null, and the official worked table confirms it. AFL does not short-circuit the unknown away.

The consequence is that a Boolean array is not really a two-valued thing. It has three states: 1, 0, and unknown. Every count, every sum, every average taken over such an array has to decide what unknown contributes — and if you do not decide, something else will.

Finding and replacing empty values

Three tools, and each says something different.

IsNull( x ) returns 1 where the value is empty. IsEmpty( ARRAY ) is a synonym; the documentation recommends IsNull in new formulas for consistency with the Null constant.

Fragment — not a complete formula

Average = MA( Close, 50 );
StillWarmingUp = IsNull( Average );      // 1 over the warm-up, 0 afterwards

Nz( x, valueifnull ) replaces empty values — and also not-a-number and infinite values — with zero, or with whatever you pass as the second argument. Since version 6.90 that second argument may itself be an array.

Fragment — not a complete formula

Signal = Nz( Close > MA( Close, 50 ) );          // unknown counts as 0
Level  = Nz( MA( Close, 50 ), Close );           // unknown falls back to the close

The documentation presents Nz() as the concise alternative to writing IIf( IsFinite( expr ), expr, 0 ), and it handles the division-by-zero infinities from lesson four in the same breath.

NullCount( array, mode ) counts them, as described above. It returns a number, so it belongs in titles, traces and if statements rather than inside a per-bar rule.

!Important

Nz() is not a way of tidying up. It is a way of writing down a decision. When you wrap a condition in Nz() you are stating that on bars where the answer is unknown, you want it treated as false. That is usually right for entry signals and usually wrong for filters that are supposed to exclude things. Ask which one you are writing before you reach for it.

Null in plots

Plot() draws nothing where an array is empty. That is why the moving average in your first formula simply started late instead of dropping to zero and drawing a cliff.

This is also a documented technique rather than merely a behaviour. The official Knowledge Base article on drawing indicators over part of a chart puts it plainly: “we simply assign Null value for the bars that we want to skip. Our graph will just be drawn for the non-null bars.” So if you want a line that appears only during a particular condition:

Fragment — not a complete formula

Average = MA( Close, 50 );
Uptrend = Close > Average;
VisibleWhenUp = IIf( Uptrend, Average, Null );
Plot( VisibleWhenUp, "MA while above", colorGreen, styleLine | styleThick );

The line vanishes wherever the condition is false, which is far cleaner than plotting zeros and then fighting the chart’s scaling.

Null in Buy and Sell — the silent failure

Here is the pattern that costs people the most, and it looks like this:

Fragment — not a complete formula

// Looks reasonable. Contains a trap.
Trend = Close > MA( Close, 200 );
Setup = RSI( 14 ) < 30;
Buy   = Trend AND Setup;
Sell  = Close < MA( Close, 50 );

For roughly the first two hundred bars of every symbol, Trend is Null, so Buy is Null. Not false. Unknown.

Three things follow, and they compound.

Your counts are wrong. Cum( Buy ) or Sum( Buy, 250 ) over an array containing unknown values does not give you the number of signals. Any statistic you compute about how often the setup occurs inherits the problem.

Your exploration or scan may behave differently from your chart, because they cover different bar ranges and therefore include different proportions of warm-up.

Symbols with short histories can be almost entirely warm-up. A company listed eighteen months ago has around 380 daily bars. If your slowest input needs 200 of them, more than half of that symbol’s life is unknown territory — and a universe scan silently applies the same formula to it as to a symbol with twenty-five years of data.

∴Research note

What the backtester does with an empty value in Buy, Sell, Short or Cover is not stated in the official documentation, and this course will not guess. The honest position is that you should never find out: write the formula so that those arrays contain only 1 and 0, and the question never arises. That is not a workaround; it is what “handle missing data explicitly” means in practice.

Two defences

Both are one line. Either is enough on its own; using both is cheap and states the intent twice.

Fragment — not a complete formula

// 1. Decide what unknown means, in the line where the ambiguity arises.
Buy = Nz( Trend AND Setup );

// 2. Refuse to answer at all until the slowest input has enough history.
WarmUpBars = 200;
HasHistory = BarIndex() >= WarmUpBars;
Buy = Nz( Trend AND Setup ) AND HasHistory;

The second is the more informative of the two, because WarmUpBars is a number a reader can check against the periods used in the formula. When a formula’s slowest ingredient is a 200-bar average and its guard says 50, the mismatch is visible.

There is one more trap in the same family, from the LastValue() documentation: LastValue() returns zero, not an error, when the last bar of the array is empty. The official example is asking for the last value of a 200-day average when only 100 days are loaded. A zero from LastValue() may mean “false”, or it may mean “there was nothing there” — and on a chart zoomed to a short window, or a newly listed symbol, the second is a real possibility.

Making it visible

This formula puts the warm-up on screen. It draws the naive condition and the guarded one side by side, and reports the length of each input’s warm-up as a number.

Complete runnable AFL

warmup-guard.afl

// warmup-guard.afl
// Part 8 - Null Values, Nz() and Warm-Up Periods
//
// Purpose:  make the warm-up period visible, show that empty values spread
//           through comparisons as well as arithmetic, and demonstrate two
//           independent defences against a condition array that is silently
//           empty at the start of the data.
// Assumes:  a symbol with clearly more than SlowPeriod bars of history, so
//           that the warm-up is a small part of the chart rather than all
//           of it.
// Apply:    Formula Editor -> Apply indicator, then scroll to the very
//           beginning of the history and compare the two shaded rows.

_SECTION_BEGIN( "Warm-up Guard" );

// ---------------------------------------------------------------------------
// Settings
// ---------------------------------------------------------------------------

SlowPeriod = 200;
FastPeriod = 20;

// ---------------------------------------------------------------------------
// Ingredients, and the empty bars they come with
// ---------------------------------------------------------------------------

SlowMA = MA( Close, SlowPeriod );
FastMA = MA( Close, FastPeriod );

// NullCount() with mode 1 reports how many consecutive empty values sit at the
// FRONT of an array. It is the cheapest way to see a warm-up as a number
// rather than as a suspicion.
SlowWarmUp = NullCount( SlowMA, 1 );
FastWarmUp = NullCount( FastMA, 1 );

// ---------------------------------------------------------------------------
// The condition, written the way most people write it first
// ---------------------------------------------------------------------------

// On every warm-up bar SlowMA is empty, so each comparison is empty too - not
// false. The AND of two empty values is empty as well. RawSignal therefore has
// a run of empty bars at the front where a reader would expect zeros.
RawSignal = Close > SlowMA AND FastMA > SlowMA;

// ---------------------------------------------------------------------------
// Two defences, either of which is enough on its own
// ---------------------------------------------------------------------------

// 1. Refuse to answer until the slowest input has enough bars behind it.
//    BarIndex() is zero-based, so bar number SlowPeriod is the first bar with
//    a complete window behind it.
HasHistory = BarIndex() >= SlowPeriod;

// 2. Turn anything still empty into an explicit zero.
Signal = Nz( RawSignal ) AND HasHistory;

// ---------------------------------------------------------------------------
// Drawing
// ---------------------------------------------------------------------------

Plot( Close,  "Close",                   colorDefault, styleCandle );
Plot( SlowMA, "MA(" + SlowPeriod + ")",  colorBlue,    styleLine | styleThick );
Plot( FastMA, "MA(" + FastPeriod + ")",  colorOrange,  styleLine );

// An array is only drawn where it has a value, so RawSignal simply stops
// existing over the warm-up while Signal is a definite zero there.
Plot( RawSignal, "RawSignal", colorRed,
      styleHistogram | styleOwnScale | styleNoLabel );
Plot( Signal, "Signal", colorGreen,
      styleHistogram | styleOwnScale | styleNoLabel );

Title = StrFormat(
    "{{NAME}} {{DATE}}   %g bars   empty at front: MA(%g) %g, MA(%g) %g   RawSignal %g   Signal %g",
    BarCount, SlowPeriod, SlowWarmUp, FastPeriod, FastWarmUp, RawSignal, Signal );

_SECTION_END();

Download warmup-guard.afl76 lines

Goal. Turn an invisible property of your data into something you can point at, and demonstrate that the two defences produce genuinely different arrays.

How it works. Two averages of very different lengths give two very different warm-ups, and NullCount( array, 1 ) reports each as a number. RawSignal is the condition written the way most people write it first, so it inherits Null from SlowMA. HasHistory uses BarIndex() — an array of bar numbers — compared against a plain number, which is scalar promotion doing its work. Signal applies both defences. The two are then plotted as separate histograms so the difference over the warm-up is visible rather than described.

Key functions. NullCount( array, mode ) counts empty values, with mode 1 meaning “at the front”. Nz( x ) replaces empty, not-a-number and infinite values with zero. BarIndex() returns the zero-based bar number as an array. IsNull( x ) is not used here but would be the way to plot the warm-up region itself.

=Expected result

Over the first stretch of the chart, the red histogram is simply absent — nothing is drawn, because there is nothing to draw — while the green one sits flat at zero. After bar 200 they become identical. The title reports the two warm-up lengths, and the longer one should be SlowPeriod - 1.

Test it. Read the two warm-up numbers from the title. MA( Close, 200 ) should report 199 leading empty values and MA( Close, 20 ) should report 19. Then click on bar 50: RawSignal should show as empty and Signal as 0. That difference, on that bar, is the entire lesson.

Common errors. If both warm-up counts read 0, the chart is showing a range that begins after the warm-up — scroll to the true start of the history or widen the range. If the two histograms look identical everywhere, check that Nz() really is wrapped around RawSignal and not around something else. If the title reports a warm-up longer than SlowPeriod, your data has gaps in it, and NullCount( SlowMA, 0 ) will tell you how many.

Extension. Add a third array, WarmUpRegion = IIf( IsNull( SlowMA ), Close, Null );, and plot it in a contrasting colour. You have now used Null deliberately, as the documented way to make a plot appear on only part of a chart — the same mechanism that was causing the problem is being used as a tool.

Recap

Null means “no value at this bar”, not zero and not false. It appears wherever a calculation lacks the history to produce an answer, and it also appears where data is genuinely missing. It propagates through arithmetic, and — the fact that catches people — through comparisons and through AND, so a Boolean array really has three states rather than two.

IsNull() finds empty values, NullCount() counts them, and Nz() replaces them with a value you choose, which is how you write down a decision rather than inherit one. Plot() draws nothing where a value is empty, which is both why your averages start late and the documented way to draw an indicator over part of a chart. And a signal array that still contains empty values is a silent failure: it corrupts counts, differs between contexts, and hits newly listed symbols hardest. Guard it, either by deciding what unknown means or by refusing to answer until there is enough history.

The last lesson of this part is about what to do when something is wrong anyway.

Check your understanding

Question 1. Cond1 is Null at bar 1 and Cond2 is 0 at bar 1. What is Cond1 AND Cond2 at bar 1?

• 0, because false AND anything is false
• Null
• 1
• An error

Show the answer and why

Answer: Null

AFL does not short-circuit the unknown away. AmiBroker’s own worked table shows exactly this case: Cond2 is 0 at bar 1, Cond1 is Null, and Buy = Cond1 AND Cond2 is Null. This is why a Boolean array has three states, not two.

Question 2. A universe scan uses a 200-bar average as a trend filter. Which symbols are most affected by the warm-up?

• The most liquid symbols
• Symbols with the longest histories
• Recently listed symbols, where the warm-up can be a large fraction of their entire history
• None; AmiBroker excludes warm-up bars automatically

Show the answer and why

Answer: Recently listed symbols, where the warm-up can be a large fraction of their entire history

A symbol listed eighteen months ago has around 380 daily bars, so a 200-bar warm-up covers more than half its life. The same formula applied across a universe therefore treats symbols very unequally, and nothing in the output says so. This is one reason Part 30 lists short-history symbols among the ways a backtest misleads.

Question 3. What does Nz( MA( Close, 50 ), Close ) produce?

• The 50-bar average, with the warm-up bars set to zero
• The 50-bar average, with the warm-up bars filled in with that bar’s closing price
• An error, because Nz() takes only one argument
• The closing price everywhere

Show the answer and why

Answer: The 50-bar average, with the warm-up bars filled in with that bar’s closing price

The optional second argument is the value used wherever the first is Null, NaN or infinite — and since version 6.90 it may itself be an array. Substituting the close is a reasonable default for a level, and a terrible one for a signal. The tool does not decide for you; that is the point.

Question 4. Which of these are true about Null in AFL? Select all that apply.

• Plot() draws nothing where an array is Null
• Assigning Null to bars you want to skip is a documented way to draw an indicator over part of a chart
• LastValue() returns zero if the final bar of the array is Null
• Null compares equal to zero

Show the answer and why

Answer: Plot() draws nothing where an array is Null, Assigning Null to bars you want to skip is a documented way to draw an indicator over part of a chart, LastValue() returns zero if the final bar of the array is Null

The first three are all documented behaviours, and the second turns the first into a technique. The fourth is the misconception the whole lesson exists to remove: Null is the absence of a value, so a comparison against it yields Null rather than a true-or-false answer.

Question 5. You wrap your entry condition in Nz() so it contains only 1 and 0. What have you actually done?

• Removed a bug from the formula
• Made a decision explicit: on bars where the answer is unknown, treat it as no signal
• Improved the accuracy of the signal
• Extended the usable history of the symbol

Show the answer and why

Answer: Made a decision explicit: on bars where the answer is unknown, treat it as no signal

Nothing about the data changed and no extra information was created. You chose an interpretation for the unknown bars and wrote it into the formula where a reader can see it. For an entry condition that choice is usually right; for a filter meant to exclude something it is often wrong, because "unknown" then becomes "permitted".

Check my answersClear