Feed Updating versus Chart Refresh

“The chart updated” describes two entirely different events, and most of the time nobody notices which one they meant. One event is new data arriving in the database. The other is a pane re-executing its formula and repainting. They run on separate clocks, either can happen without the other, and almost every “my real-time chart is broken” report is really a report that one of the two stopped while the other carried on.

By the end of this lesson you should be able to name which of the two has failed from what you can see on screen, state what makes a pane redraw, read the plugin’s connection state and know what each state means, describe what backfill does after a reconnection, and explain the case that catches everybody: a chart that is completely up to date and completely wrong.

Two clocks

Follow one trade from the exchange to your eye and the separation is obvious.

From a trade to a repainted pane

1. 1A trade prints at the exchangeNothing to do with AmiBroker yet
2. 2The vendor distributes itYour subscription decides whether you receive it at all
3. 3The plugin receives itIt packs incoming ticks into base-interval bars
4. 4The database is updatedThe DATA clock ticks here
5. 5A refresh is duePreferences interval, RequestTimedRefresh, or something you did
6. 6The pane re-executes its whole formulaThe PAINT clock ticks here
7. 7Pixels changeWhat you actually observe

Two independent ticks. Observing the last step tells you only that the second one happened.

The consequences of the separation are not symmetrical, and it is worth being precise about each direction.

The paint clock can run while the data clock is stopped. This is the dangerous one, because it looks healthy. The pane redraws on schedule, the title updates, any clock you print in the title advances — and the last bar has not changed since the feed died. Nothing on a default chart makes this visible.

The data clock can run while the paint clock is stopped. This is the annoying one but the harmless one: the database is receiving everything and the picture is behind. Your data is intact; you are just looking at an old drawing of it.

What actually makes a pane redraw

There is no single refresh mechanism. There are four, and they stack.

1. The real-time chart refresh interval. Tools → Preferences → Intraday carries an interval between automatic chart refreshes in real-time mode, documented with a default of 3 seconds, which you may drop to 1 in a volatile market. Entering zero switches to every-tick refreshing: AmiBroker then refreshes charts on every arriving trade, provided the formulas execute fast enough, and dynamically throttles itself to stay under about 50% average CPU — so panes taking 0.2 s each refresh roughly 2.5 times per second. The manual states plainly that the Standard edition will not allow zero to be entered.

✱AmiBroker Professional required

2. RequestTimedRefresh( interval, onlyvisible ). A per-pane request that makes this pane re-execute every interval seconds, documented as working “regardless of data source used or connection state”. That last clause is the key to this whole part: it works with no plugin, on an end-of-day database, offline. Refreshes are aligned to second boundaries, so RequestTimedRefresh( 5 ) fires at seconds 0, 5, 10 and so on. Part 23 covers it properly; here we need only its most useful property, which is that it lets the paint clock run when the data clock cannot.

3. Things you do. Changing symbol or interval, scrolling, zooming, resizing, selecting a quote, moving the mouse over a pane that called RequestMouseMoveRefresh().

4. New data itself, subject to the interval in point 1.

Status( "redrawaction" ) tells a formula which kind of refresh it is in: 0 for a regular refresh (user action or new data) and 1 for a timer refresh raised by RequestTimedRefresh(). That single value is what stops a formula from re-sending an alert every time you nudge the mouse.

✕Common mistake

onlyvisible defaults to True, and the documentation is explicit that this applies to the main AmiBroker window too: when the main window is minimised, charts are not refreshed. Passing False fixes that. What False does not fix is a pane sitting on an inactive chart sheet — those panes “do not really exist until they are shown”, so they cannot be refreshed at all. Anyone who builds a monitoring sheet they never look at and wonders why it never fires has met this rule.

The refresh asymmetry that generates the most support questions

The Real-Time Quote window is documented as refreshing at least ten times per second. A chart formula sees a new value only when the pane re-executes, which on the default setting is every three seconds. So GetRTData("Bid") in a chart formula and the bid in the quote window will legitimately disagree at any given instant, and the chart will be the stale one. That is documented, expected behaviour rather than a fault — and it is a good reason to treat the quote window, not your own panel, as the reference display when you are checking whether the feed is alive.

Reading the plugin’s connection state

AmiBroker publishes the connection state in the plugin status area at the lower right of the main window. When it changes, AmiBroker beeps and pops a bubble tooltip with more detail; the tooltip disappears after two seconds and you can bring it back by hovering. The documented states:

State	Light	What it means	What you do
OK	Green	Connection fine, plugin operating correctly	Nothing
WAIT	Yellow	Connection being set up, or connected to only some of several servers. Usually transient	Wait a few seconds
ERR	Red	Connection broken: wrong username or password, or a required third-party component is not running	Fix the cause; the plugin then reconnects by itself
SHUT	Purple	A serious problem occurred and the plugin will not attempt to reconnect	Fix the cause, then reconnect by hand from the plugin context menu, or restart AmiBroker

The distinction between ERR and SHUT is the one worth memorising. ERR recovers on its own once the cause is removed. SHUT does not, ever, and a machine left running overnight in SHUT will present you with a chart in the morning that has been repainting faithfully for fourteen hours without receiving a single quote.

Right-clicking that status area opens the plugin context menu. Plugins differ, but the guide says most offer at least Reconnect, Shutdown (Disconnect) and Force backfill.

!Important

No AFL function reports the plugin connection status. Nothing in the function reference exposes OK / WAIT / ERR / SHUT, and nothing lets a formula force a reconnect or a backfill — Force backfill is a context-menu action and “Wait for backfill” is an Analysis-window checkbox. This is a limit, not an oversight to work around: your AFL can detect that data stopped arriving, and it cannot detect why. Design your monitoring around that division.

Backfill, and what a reconnection actually repairs

Backfill is the plugin downloading history rather than streaming it. In normal operation it is automatic: the guide states that if the plugin detects missing quotes between the last available bar and the current date and time, it triggers a backfill itself.

Force backfill re-downloads the entire intraday history for a symbol, and the guide names exactly two situations where you should reach for it: after you have enlarged “number of bars to load” in Database Settings, since symbols backfilled under the smaller number stay small; and to clean up bad ticks, in the hope that the vendor has repaired its own history — noted as working well for eSignal, which does appear to fix bad ticks.

Wait for backfill is a checkbox in the Analysis window, available for the eSignal, myTrack and IQFeed plugins, which makes scans, explorations and backfills wait for each symbol’s backfill to finish. It has no effect on databases without plugins, on end-of-day plugins, or with the QuoteTracker plugin. The edition table lists it as Professional only. The guide’s own recipe for backfilling a whole database is worth knowing: send Buy = 1; to the Analysis window, set Apply to to all symbols and Range to one recent bar, tick “Wait for backfill”, and press Scan.

⌀Data quality warning

After a reconnection, the last three bars of an intraday symbol cannot be edited in the Quote Editor, because they are cached inside the plugin. They become editable only once newer bars arrive and push them back. If you are trying to repair a gap by hand and the editor refuses, that is why — and the correct fix is a forced backfill rather than manual editing.

Why a chart can be current and wrong

“Current” and “correct” are different properties. Here are five documented ways they come apart, each with the observation that distinguishes it.

The bar is still forming. The newest bar’s high, low and close will keep changing until the interval ends. A condition can go true and then false again inside a single bar. This is not a defect; it is what a partial bar is. Part 25 gives the documented remedy for alerts — evaluate on completed bars only.

The symbol is not really streaming. You may add more tickers to the database than your subscription allows, and AmiBroker will rotate them so the most recently used stay active while older ones are dropped from the Data Manager. Exceeding your limit significantly “may lead to some problems”. The Real-Time Quote window is documented as not participating in that rotation and as unable to hold more symbols than your subscription limit — which makes it a useful test: if a symbol updates in the quote window and not on the chart, the feed is fine.

A backfill is rewriting history underneath you. During a backfill the array your formula sees is changing for reasons that have nothing to do with new trades.

The timestamps are wrong rather than the prices. If the database time shift is misconfigured, every bar arrives promptly and lands in the wrong place on the time axis. Status("timeshift") reports the shift in seconds, and Part 20 is entirely about the damage this does.

Bar Replay is still running. Bar Replay is global: it truncates data for all symbols at the playback position and affects charts and the Analysis window alike. A replay left in PAUSE looks exactly like a frozen feed. GetPlaybackDateTime() returns the playback position, or zero when replay is not active, which makes this the easiest of the five to test from AFL.

Seeing both clocks at once

Goal

Build the smallest possible instrument that separates the two clocks: a pane that counts its own executions on one line and reports the newest bar’s timestamp on the next. When the two disagree you can say immediately which half of the system stopped. It needs no subscription to be useful — in fact it is most instructive with no feed at all.

Complete formula

Complete runnable AFL

refresh-vs-update.afl

_SECTION_BEGIN( "Two clocks" );

/*  Two clocks - Part 21, Real-Time Charts.

    WHAT IT DOES
        Separates the two events that people both describe as "the chart
        updated":
            the PAINT clock - this pane re-executed its formula and redrew;
            the DATA clock  - new data reached the database.
        The panel counts its own executions on the left and shows the newest
        bar's timestamp on the right. On a database with no feed attached the
        first number climbs every second and the second one never moves.
        That is the entire lesson, and seeing it costs nothing.

    ASSUMPTIONS AND LIMITS
        - RequestTimedRefresh() works on any database, with or without a
          real-time plugin, and regardless of connection state. That is what
          makes this demonstration possible offline.
        - Status( "lastrtupdate" ) is documented as the date/time of the last
          update sent by the real-time plugin. Nothing in the documentation
          promises what it returns when no plugin is feeding the database, so
          the code below tests it for equality with zero and treats anything
          else as "no stamp available" rather than assuming a sentinel.
        - Now( 5 ) is the PC clock. It is not exchange time and it is not the
          bar timestamp. If the database time shift is non-zero the two do not
          agree; Status( "timeshift" ) reports the difference in seconds.
        - DateTime values are a bitset. They may be compared with == and !=
          only. Every ordering here goes through DateTimeDiff().
        - Status( "lastbarend" ) works for time-based bars only.
*/

RefreshSeconds = Param( "Refresh interval (seconds)", 1, 1, 60, 1 );

// False keeps the pane refreshing even when the main window is minimised.
// It has no effect on panes sitting on an inactive chart sheet: those panes
// do not exist until they are shown, so they cannot be refreshed at all.
RequestTimedRefresh( RefreshSeconds, False );

// One counter per chart pane, so two copies of this panel do not share state.
CounterKey = "p21_paintcount_" + NumToStr( GetChartID(), 1.0, False );
PaintCount = Nz( StaticVarGet( CounterKey ) ) + 1;
StaticVarSet( CounterKey, PaintCount );

// 0 = ordinary refresh (user action, new data, resize); 1 = our timer.
if( Status( "redrawaction" ) == 1 )
    TriggerText = "timer";
else
    TriggerText = "user action, new data or resize";

LastBarStamp = LastValue( DateTime() );
BarEnd       = Nz( Status( "lastbarend" ) );
BarLeft      = Status( "lastbartimeleft" );
RtUpdate     = Nz( Status( "lastrtupdate" ) );
TimeShift    = Status( "timeshift" );
Playback     = GetPlaybackDateTime();      // zero when Bar Replay is inactive

if( RtUpdate != 0 )
    FeedText = "plugin last update " + DateTimeToStr( RtUpdate ) + "  ("
               + NumToStr( DateTimeDiff( Now( 5 ), RtUpdate ), 1.0 ) + " s ago)";
else
    FeedText = "no plugin update stamp on this database";

if( BarEnd != 0 )
    BarEndText = DateTimeToStr( BarEnd ) + "  ("
                 + NumToStr( DateTimeDiff( Now( 5 ), BarEnd ), 1.0 ) + " s ago)";
else
    BarEndText = "unavailable - not a time-based interval";

if( Playback != 0 )
    ReplayText = EncodeColor( colorOrange )
                 + "Bar Replay ACTIVE at " + DateTimeToStr( Playback );
else
    ReplayText = EncodeColor( colorDefault ) + "Bar Replay inactive";

SetChartOptions( 2, chartWrapTitle );
Plot( Close, "Close", colorDefault, styleCandle );

Title =
    EncodeColor( colorDefault ) + Name() + "   " + Interval( 2 ) + "\n"
  + "PAINT clock - executions since this pane was loaded: "
  + NumToStr( PaintCount, 1.0 ) + "   (this one triggered by " + TriggerText + ")\n"
  + "PAINT clock - system time now: " + Now( 0 ) + "\n"
  + "DATA clock - newest bar stamped: " + DateTimeToStr( LastBarStamp ) + "\n"
  + "DATA clock - that bar ends: " + BarEndText + "\n"
  + "DATA clock - seconds left in the forming bar: " + NumToStr( BarLeft, 1.0 ) + "\n"
  + "DATA clock - " + FeedText + "\n"
  + "Database time shift: " + NumToStr( TimeShift, 1.0 ) + " s\n"
  + ReplayText;

_SECTION_END();

Download refresh-vs-update.afl90 lines

How it works

RequestTimedRefresh( RefreshSeconds, False ) starts the paint clock and keeps it running even when the main window is minimised. Each execution increments a counter held in a static variable whose name includes GetChartID(), so two copies of the panel in different panes keep separate counts instead of fighting over one.

Everything below that reads the data clock from four different angles, and each is guarded rather than assumed:

• LastValue( DateTime() ) — the timestamp of the newest bar in the array.
• Status( "lastbarend" ) — the moment that bar’s interval ends. A 5-minute bar at 09:00 ends at 09:04:59. Time-based bars only, so it is checked before use.
• Status( "lastbartimeleft" ) — seconds remaining in the forming bar.
• Status( "lastrtupdate" ) — the date and time of the last update sent by the real-time plugin.

The documentation does not promise what Status("lastrtupdate") returns on a database with no plugin, so the formula compares it with zero for equality — the one comparison the documentation says is safe on DateTime values — and prints an honest “no plugin update stamp” when there is nothing to report. Every age in the panel goes through DateTimeDiff(), because DateTime values are a bitset and ordinary > and < comparisons on them are documented as unreliable.

Key functions

• RequestTimedRefresh( interval, onlyvisible = True ) — per-pane timed re-execution, independent of the data source.
• Status( "redrawaction" ) — 0 for a regular refresh, 1 for a timer refresh.
• StaticVarSet / StaticVarGet — values that survive between executions of the same formula, which is the only way a chart pane can remember anything.
• Nz( x ) — turns Null into zero so a missing value cannot silently propagate.
• DateTimeDiff( a, b ) — difference in seconds, positive when a is later.
• GetPlaybackDateTime() — Bar Replay position, or zero when replay is inactive.

Expected result

On a live feed during market hours: the execution counter climbs once per second, the newest bar timestamp advances at the interval of the chart, the seconds-left countdown decreases and resets, and the plugin update stamp reads a small number of seconds ago.

On any database with no feed — the Level A case — the counter climbs exactly the same way, the system time advances, and the newest bar timestamp never moves at all. The panel reports “no plugin update stamp on this database” rather than pretending. That contrast, on screen, is the entire lesson.

Test it

1. Load it on a daily end-of-day database. Confirm the counter increments and the bar timestamp does not. You have now seen the paint clock without the data clock.
2. Minimise AmiBroker for thirty seconds, restore it, and check the counter. With False as the second argument it should have kept counting. Change it to True, repeat, and watch the counter stall while minimised.
3. Put the panel on a second chart sheet, work on the first sheet for a minute, then switch. The counter will not have advanced while the sheet was hidden — inactive-sheet panes do not exist.
4. Open Tools → Bar Replay, set a range and press PAUSE. The last line should switch to “Bar Replay ACTIVE” and show the playback position.

Common errors

• Reading the execution counter as a data-freshness measure. It measures the opposite: it proves the pane is alive, which is precisely what makes a frozen bar timestamp meaningful.
• Expecting the counter to survive a formula edit. Saving the formula reloads it and the static variable keeps counting from wherever it was — remove it with StaticVarRemove if you want a clean start.
• Comparing Now(5) with a bar timestamp on a database whose time shift is not zero. The two are in different frames; the panel prints the shift so you can see it.

Extension

Add a second static variable that records the previous newest-bar timestamp and count how many refreshes have passed since it last changed. That counter — refreshes since the data last moved — is the seed of the detector built at the end of this part.

If you have no live feed

Everything in this lesson except the plugin status area is reachable at Level A, and the most important idea in it is easier to see without a feed than with one.

• The two-clock panel runs on any database, including a daily end-of-day one. With no feed attached the data clock is permanently stopped, so the separation between the clocks is maximally visible.
• To watch the data clock move without a feed, use Tools → Bar Replay on an intraday database of historical bars. Press PLAY and the newest-bar timestamp advances step by step while the panel’s replay line shows the playback position, which is exactly the pattern a live feed produces.
• The plugin connection states are the one part you cannot reproduce. Learn the table above, and note the one fact that matters most and needs no feed to remember: SHUT never reconnects by itself.

Recap

The feed writing to the database and the pane redrawing are separate events on separate clocks. Redraws come from the Preferences interval, from RequestTimedRefresh(), from your own actions and from new data, and Status("redrawaction") tells a formula which. The plugin’s connection state lives in the status area as OK, WAIT, ERR or SHUT, and only SHUT refuses to recover by itself. Backfill repairs gaps automatically, and a forced backfill exists for the two documented cases where automatic repair is not enough. A chart can be perfectly current and still wrong — because the bar is unfinished, because the symbol was rotated out, because a backfill is in flight, because the time shift is wrong, or because Bar Replay never stopped.

The lab now builds a layout on this foundation, and the challenge that follows turns the diagnosis into a procedure.

Check your understanding

Question 1. A chart pane calls RequestTimedRefresh( 1 ) and its execution counter is climbing once per second, but the newest bar has read 10:47 for twenty minutes. What has failed?

• The chart refresh mechanism
• The data clock — nothing new is reaching the database — while the paint clock is healthy
• The formula, since it should not execute when there is no new data
• Nothing; a bar only updates when the interval completes

Show the answer and why

Answer: The data clock — nothing new is reaching the database — while the paint clock is healthy

A climbing counter proves the pane is executing and repainting. A frozen newest bar proves no new data arrived. The two together isolate the fault to the data side — feed, plugin, subscription or session — and rule out the chart entirely.

Question 2. Which of these will keep a pane refreshing while the main AmiBroker window is minimised?

• RequestTimedRefresh( 5 )
• RequestTimedRefresh( 5, False )
• Setting the Preferences refresh interval to zero
• Placing the pane on a separate chart sheet

Show the answer and why

Answer: RequestTimedRefresh( 5, False )

The onlyvisible parameter defaults to True, and the documentation states that this covers the main window being minimised. Passing False keeps refreshes coming. It does not help with panes on inactive sheets, which do not exist until shown and therefore cannot be refreshed at all.

Question 3. The plugin status area shows SHUT in purple. What does the User’s Guide say happens next?

• The plugin retries automatically once the underlying problem is resolved
• AmiBroker restarts the plugin after a timeout
• The plugin will not attempt to reconnect; you must reconnect from the plugin context menu or restart AmiBroker
• Charts stop refreshing until the connection returns

Show the answer and why

Answer: The plugin will not attempt to reconnect; you must reconnect from the plugin context menu or restart AmiBroker

ERR is the state that recovers by itself once the cause is fixed. SHUT is explicitly documented as not attempting reconnection. Charts keep refreshing throughout, which is exactly why a SHUT state can go unnoticed for hours.

Question 4. Which observations would help distinguish "the feed has stopped" from "this one symbol has stopped"? Select all that apply.

• Whether other symbols in the Real-Time Quote window are still moving
• Whether the pane’s own execution counter is still climbing
• Whether the symbol count you have added exceeds your subscription limit
• Whether the chart is drawn as candlesticks or bars

Show the answer and why

Answer: Whether other symbols in the Real-Time Quote window are still moving, Whether the symbol count you have added exceeds your subscription limit

The quote window separates "this symbol" from "everything", and the documented symbol-rotation behaviour explains why one symbol can go quiet while the feed is healthy. The execution counter distinguishes paint from data, not one symbol from another, and the chart style is irrelevant.

Check my answersClear