The Real-Time Architecture, End to End

An intraday chart that has stopped updating has at least seven possible causes, and only one of them is AmiBroker. Working out which one you are looking at is not guesswork if you know the chain, and it is nothing but guesswork if you do not. So before any configuration, any plugin and any live formula, this lesson draws the chain.

By the end you should be able to name every link between the trade that happened and the bar you are looking at, say what each link adds to the delay, say what each link looks like when it fails, and — this is the part people skip — say which links belong to AmiBroker and which belong to somebody else entirely.

The chain, drawn

From the trade to your chart pane

data flows down

1. The venueAn exchange or other trading venue matches a buyer and a seller. This is the only place in the diagram where a price is creatednot yours
2. The venue's own data feedThe venue publishes its trades and quotes on its own terms, at its own speed, under a licence it writespriced by the venue
3. A vendor or a brokerCollects one or more venue feeds, normalises them, decides what history to keep, what symbol names to use and who may receive whatyou subscribe
4. The vendor's program on your PCIQConnect, Trader Workstation, the eSignal Data Manager. Every real-time plugin documented in the User's Guide talks to a program on your own machine, not to the vendor over the internetmust be running
5. The AmiBroker data pluginA DLL in AmiBroker’s Plugins folder that translates one vendor’s protocol into the requests AmiBroker makes
6. The AmiBroker databaseBars stored at a fixed base interval, plus your symbols, categories, watch lists and studiesyou configure
7. AFL, charts, Analysis, alertsEverything you build, re-executed on a schedule you choose

Seven links. AmiBroker's territory starts at the plugin boundary; everything above it belongs to organisations you do not control.

Read it downwards for data and upwards for blame. A price is created at the top. It becomes yours at the bottom. In between, four organisations that have never heard of each other each get to decide something about it.

AmiBroker is not the exchange, and usually not your data source

This sounds obvious written down and is routinely forgotten in practice, so it is worth making concrete. AmiBroker sells software. It does not sell market data, does not operate a feed, and has no view about what the correct price of anything is. The two things that look like exceptions are not: the sample database of Dow Jones components that ships with the installer is a starter file, and AmiQuote is a separately licensed downloader that fetches from other people’s public sources.

Three consequences follow, and each of them changes who you should be talking to when something looks wrong.

The vendor decides what exists. Which instruments you can see, how deep the history goes, whether options or futures are included, what a symbol is called, whether bid and ask sizes are populated at all — every one of those is the vendor’s decision, expressed through the plugin. AmiBroker’s own function reference for GetRTData() makes the point explicitly: which fields carry data depends on the underlying source, and the recommended way to find out is to look at the Real-Time Quote window and see which columns are populated.

The plugin’s access is read-only. The User’s Guide states this plainly: data plugins provide read-only access to the vendor’s data, and AmiBroker never writes back to the external source. You cannot correct a bad tick at the vendor from inside AmiBroker. You can ask for a fresh copy — that is what Force backfill does — and if the vendor has repaired its history you get the repair, and if it has not, you get the same bad tick again.

Everything that is yours is still stored by AmiBroker. Studies, drawing objects, category, group and watch list assignments, composites and favourites live in AmiBroker’s own database files even when a plugin supplies every quote. A database fed by a plugin is still an AmiBroker database that you create, configure and maintain — a point learners miss so consistently that the guide says it twice.

✕Common mistake

Believing that choosing an external data source means “no AmiBroker database needed”. The symptom is a reader who never sets Number of bars to load, never chooses a base interval deliberately, and then cannot explain why their intraday chart holds three weeks of history when the vendor offers a year. The plugin supplies quotes; the database still decides how many of them are kept and at what grain.

What a data plugin actually does

A data plugin is a DLL sitting in the Plugins subfolder of the AmiBroker program directory. AmiBroker scans that folder at startup, and Tools -> Plugins lists what it found, with version numbers — which is the documented way to confirm which build of a plugin you are actually running after replacing one by hand.

Selecting it is a database-level decision, not a formula-level one. File -> Database Settings has a Data source combo; (local) means AmiBroker maintains the quote files itself, and anything else hands that job to a plugin. From that moment the plugin owns the quotation data for that database, and four behaviours follow that have no equivalent in a local database:

• It answers requests rather than holding a file. AmiBroker asks the plugin for a symbol’s bars; the plugin gets them from the vendor’s client program and hands them back. Local data storage, also in Database Settings, controls whether AmiBroker keeps its own cached copy of what came back. The guide describes enabling it as required for most real-time sources.
• It backfills on demand, per symbol. History arrives the first time something touches a symbol — you open its chart, or an Analysis run reaches it. Nothing pre-loads the whole database unless you make it.
• It manages a subscription you do not see. The guide is explicit that you may add more tickers to the database than your feed subscription allows, and that AmiBroker rotates the active ones so the most recently used stay live. It is equally explicit that going significantly beyond your limit “may lead to some problems”, and that the rotation does not apply to the Real-Time Quote window, which cannot hold more symbols than your subscription permits.
• It reports a connection state. A small status area at the lower right of the main window shows the plugin’s view of its own health. The next lesson but one takes that seriously; for now, note that it exists and that it is the only place the connection state is visible.

⚠Warning

Plugins are third-party code running inside your AmiBroker process. The Plugins window documentation carries an official warning that AmiBroker makes no representations about non-certified third-party plug-ins, that they can cause instability or crashes, and that using them is at your own risk. Plugins are also DLLs, so bitness must match: a 32-bit AmiBroker cannot load a 64-bit plugin, and a plugin that fails to appear in the data source list is a bitness mismatch until proven otherwise.

Where the time goes

Every link adds delay, and the delays are not all of the same kind. Some are physical, some are commercial, and at least one is a setting on your own machine that most people never look at.

Hop	What adds time	Whose decision
Venue to venue feed	Matching, publication and the venue’s own distribution	The venue
Venue feed to vendor	Network distance, and the vendor’s own processing and normalisation	The vendor
Vendor to your client program	Your connection, plus any contractual delay attached to a delayed product	Your subscription
Client program to plugin	The vendor’s local API. Interactive Brokers’ documentation describes what Trader Workstation streams as 0.2 to 0.3 second snapshots rather than every tick	The vendor’s software
Plugin to database	Time-compression of incoming updates into bars at the base interval	Your base interval
Database to chart	The real-time chart refresh interval, which AmiBroker documents as three seconds by default	You
Chart to your eye	How long the formula takes to run, once per refresh, per pane	Your code

Two documented asymmetries are worth carrying around, because they cause arguments.

The first is that different parts of AmiBroker see different freshness. The GetRTData() reference states that the built-in Real-Time Quote window refreshes at least ten times a second, while a formula only sees a new value when its pane re-executes. A chart on the default three-second refresh interval therefore shows a bid that lags the quote window. That is expected behaviour, not a defect, and it is the most common “my AFL bid is stale” complaint.

The second is that timers are approximate. RequestTimedRefresh() is documented as accurate to roughly ±55 milliseconds when the machine is not busy, with the first line of your code typically executing somewhere in the first hundred milliseconds of the second. Windows is not a real-time operating system and offers no fixed reaction time. Nothing you build in this course should depend on firing at an exact instant.

∴Research note

No number in the table above is a measurement of your setup, and this course will not invent one. Latency depends on your vendor, your distance from their servers, your connection and your machine. The two figures quoted — Interactive Brokers’ 0.2 to 0.3 second snapshots and AmiBroker’s three-second default chart refresh — come from official documentation and are stated as documentation, not as a forecast of what you would measure. Measuring your own is a worthwhile afternoon; the panel later in this lesson is where to start.

What breaks at each link

The awkward property of this chain is that most failures look identical from the bottom: the last bar stops changing. Below is the same chain again, read as a fault table.

Link	What you see	Where to look first
Venue	Nothing changes because nothing traded — a holiday, a half day, a halt, or a genuinely quiet instrument	Another symbol on the same venue. If it is moving, the feed is fine
Entitlement	One symbol or one whole exchange stays blank while everything else updates	The vendor’s entitlement list for your account, not AmiBroker
Vendor client program	The plugin status light shows ERR. The documented causes are invalid credentials or a required third-party program that is not running	Is Trader Workstation, IQConnect or the eSignal Data Manager actually running and logged in?
Plugin connection	The status light shows WAIT while connecting, or SHUT after a serious failure — and SHUT does not retry by itself	Reconnect from the plugin status context menu, or restart AmiBroker
Subscription limits	Some symbols update and others do not, apparently at random	How many symbols you have streaming against how many you pay for
Database	Bars exist but are missing from the chart, or the chart cannot show the interval you want	Intraday Settings filtering, which hides data rather than deleting it, and the base interval
Formula or pane	Everything else is current and this one pane is not	The refresh interval, and whether the pane is on a sheet that is actually visible

That last row deserves its own warning, because it is invisible by construction.

✕Common mistake

Building an alert or scanner pane on a chart sheet you never look at. RequestTimedRefresh() has an onlyvisible argument that defaults to True, and setting it to False keeps minimised windows refreshing — but the documentation is explicit that panes sitting on inactive sheets do not really exist until they are shown, and cannot be refreshed at all. Turning onlyvisible off does not resurrect them. The symptom is an alert formula that has never fired once and looks perfectly correct.

Reading the chain from inside AFL

Goal

A chart pane that tells you which link you are looking at. Not whether the market is moving — the chart already shows that, and misleadingly. The panel reports three clocks and the distance between them: your computer’s clock, the timestamp of the newest bar AmiBroker holds for this symbol, and the time of the last update a real-time plugin reported. When a chart stops, exactly one of those three has stopped, and knowing which one converts a mystery into a two-minute check.

The formula

Complete runnable AFL

data-path-report.afl

// ===========================================================================
//  Data path report
//
//  WHAT IT IS FOR
//    A chart pane that puts the three clocks of a real-time setup side by side
//    and measures the distance between them:
//      1. this computer's clock,
//      2. the timestamp of the newest bar AmiBroker holds for this symbol,
//      3. the time of the last update a real-time plugin reported.
//    When a chart appears to have stopped, one of those three has stopped
//    moving, and which one tells you which link in the chain to go and inspect.
//
//  HOW TO RUN IT
//    Formula Editor -> Apply Indicator, into its own chart pane.
//    It runs on any database, with or without a real-time feed. On an
//    end-of-day database it reports that no plugin update was seen, which is
//    the correct answer for that database, not a failure of the formula.
//
//  ASSUMPTIONS, STATED SO THEY CAN BE CHECKED
//    - Now(5) is the PC clock. Bar timestamps come out of the database and may
//      be shifted towards exchange time, so the two are not necessarily
//      measured on the same clock. The database timeshift is printed as well
//      so that the size of that discrepancy is visible rather than hidden.
//    - The User's Guide documents Status("lastrtupdate") as the date/time of
//      the last update sent by a real-time plugin. It does not document what
//      the call returns on a database that has no plugin attached, so this
//      formula accepts the value only when it is positive and yields an age
//      inside a plausible band, and otherwise reports "not seen".
//    - Status("lastbartimeleft") is documented for time-based bars only and
//      relies on the database timeshift being correct, so it is shown only
//      when it comes back positive.
//    - The staleness threshold below is a choice you make about your own
//      tolerance. It is not a statement about any vendor's service level.
// ===========================================================================

_SECTION_BEGIN( "Data path report" );

// Both of these are judgements rather than facts, so they live at the top
// where they can be seen and changed.
RefreshSeconds = Param( "Refresh every (seconds)", 2, 1, 60, 1 );
StaleAfterSecs = Param( "Call the feed stale after (seconds)", 30, 5, 600, 5 );

// Re-execute this pane on a timer. Every refresh runs the entire formula
// again, so this is the one setting in the file with a CPU cost attached.
RequestTimedRefresh( RefreshSeconds );

// --- Clock 1: this machine -------------------------------------------------
PcClock = Now( 5 );

// --- Clock 2: the newest bar the database holds for the selected symbol ----
NewestBar     = LastValue( DateTime() );
BarAgeSeconds = DateTimeDiff( PcClock, NewestBar );

// --- Clock 3: the last thing a real-time plugin said -----------------------
// Anything that is not a positive timestamp yielding an age between "a minute
// in the future" and "thirty days ago" is treated as no report at all, rather
// than being printed as though it meant something.
LastRtUpdate = Status( "lastrtupdate" );
RtAgeSeconds = DateTimeDiff( PcClock, LastRtUpdate );
SawRtUpdate  = LastRtUpdate > 0 AND RtAgeSeconds > -60 AND RtAgeSeconds < 30 * 86400;

// --- Bar Replay, which quietly overrides everything above ------------------
// GetPlaybackDateTime() returns zero when replay is not running, so the test
// has to be explicit or the zero prints as a date in 1899.
PlaybackPosition = GetPlaybackDateTime();
ReplayRunning    = PlaybackPosition > 0;

TimeShiftHours   = Status( "timeshift" ) / 3600;
SecondsLeftInBar = Status( "lastbartimeleft" );

// Interval( 2 ) is the readable name of the CHART interval, not of the
// database base interval - AFL has no documented way to report the latter.
// Display it, but never compare it as text: use Interval() in seconds.
ReportText =
      "DATA PATH REPORT for " + Name() + "\n"
    + "Chart interval: " + Interval( 2 )
    + "  (" + NumToStr( Interval(), 1.0 ) + " seconds per bar)\n"
    + "Database timeshift: " + NumToStr( TimeShiftHours, 1.2 ) + " hours\n"
    + "\n"
    + "PC clock now:      " + DateTimeToStr( PcClock ) + "\n"
    + "Newest bar held:   " + DateTimeToStr( NewestBar )
    + "   (" + NumToStr( BarAgeSeconds / 60, 1.1 ) + " minutes behind the PC clock)\n";

if( SawRtUpdate )
    ReportText = ReportText
        + "Last plugin update: " + DateTimeToStr( LastRtUpdate )
        + "   (" + NumToStr( RtAgeSeconds, 1.0 ) + " seconds ago)\n";
else
    ReportText = ReportText + "Last plugin update: not seen\n";

if( SecondsLeftInBar > 0 )
    ReportText = ReportText
        + "Current bar completes in " + NumToStr( SecondsLeftInBar, 1.0 ) + " seconds\n";

// The verdict is deliberately about which link to inspect, not about whether
// anything is wrong. A quiet feed and a broken feed look identical from here.
if( ReplayRunning )
{
    Verdict = "BAR REPLAY IS ACTIVE (position "
            + DateTimeToStr( PlaybackPosition )
            + "). Every bar above is a replayed bar. Press STOP in the Bar "
            + "Replay window before reading any of this as live.";
    VerdictColour = colorOrange;
}
else
{
    if( NOT SawRtUpdate )
    {
        Verdict = "No real-time plugin update has been reported. Either this "
                + "database is not fed by a streaming plugin, or the plugin "
                + "has delivered nothing since AmiBroker started.";
        VerdictColour = colorLightGrey;
    }
    else
    {
        if( RtAgeSeconds > StaleAfterSecs )
        {
            Verdict = "The plugin last reported "
                    + NumToStr( RtAgeSeconds, 1.0 )
                    + " seconds ago, which is past your threshold of "
                    + NumToStr( StaleAfterSecs, 1.0 )
                    + " seconds. Check the plugin status light before "
                    + "concluding the market is quiet.";
            VerdictColour = colorRed;
        }
        else
        {
            Verdict = "The plugin reported within your threshold. That says "
                    + "the feed is arriving; it says nothing about whether "
                    + "the prices in it are correct.";
            VerdictColour = colorGreen;
        }
    }
}

Title = ReportText + "\n" + Verdict;

// The pane is about the text above; the price line is here only so that the
// pane has something to scale against and can sit under a price chart.
Plot( Close, "Close", VerdictColour, styleLine | styleNoTitle );

_SECTION_END();

Download data-path-report.afl142 lines

How it works

The formula has four parts and no cleverness.

It begins by asking to be re-run. RequestTimedRefresh( RefreshSeconds ) makes the pane re-execute on a timer, and the documentation notes that this works regardless of the data source and regardless of whether anything is connected — which is exactly why the panel is usable at Level A.

It then collects three timestamps. Now( 5 ) returns the PC clock as a DateTime. LastValue( DateTime() ) returns the timestamp of the last bar in the array, which is the database’s answer to “how recent is my data”. Status( "lastrtupdate" ) returns the time of the last update a real-time plugin sent, which is the plugin’s answer to the same question. The three are compared with DateTimeDiff(), never with > or <, because DateTime values are documented as a bitset for which only equality comparisons are reliable.

It guards the plugin value rather than trusting it. The official page documents what Status( "lastrtupdate" ) returns when a plugin is present; it does not document what it returns when no plugin is attached. Rather than inventing a rule, the formula accepts the value only when it is positive and yields an age inside a plausible band, and otherwise prints “not seen” — which is the honest description of the situation on an end-of-day database.

Finally it checks whether Bar Replay is driving the chart. GetPlaybackDateTime() returns the playback position, or zero when replay is not active, so the test has to be explicit; printed unguarded, that zero renders as a date in the nineteenth century.

Functions you may not have met

• Status( "lastrtupdate" ) — the DateTime of the last update sent by a real-time plugin. Related codes on the same page: "timeshift", the database timeshift in seconds, and "lastbartimeleft", the seconds remaining in the forming bar, documented for time-based bars only and dependent on the timeshift being set correctly.
• DateTimeDiff( a, b ) — the difference between two DateTime values in seconds, positive when the first is later. This is the documented way to compare them.
• RequestTimedRefresh( interval, onlyvisible ) — re-executes this pane every interval seconds. Part 23 goes into what it costs.
• GetPlaybackDateTime() — the Bar Replay position, or zero when replay is off.
• Interval( 2 ) — the name of the chart interval as text. Display it; never compare it as text, because the names are translated in localised builds. Use Interval() in seconds for comparisons.

What you should see

=Expected result

On an end-of-day database with no plugin: the chart interval, a timeshift of zero hours, your PC clock, a newest bar somewhere between a few hours and a few days old, and “Last plugin update: not seen”. The verdict line reports that no plugin update has been reported and the price line is drawn in grey. That is the correct output, and reading it teaches the same lesson as the live case.

On a plugin-fed database during market hours: the same panel with a plugin update timestamped seconds ago, and a green verdict. Outside market hours, the same panel with a plugin update that gets steadily older — which is what a quiet market looks like from inside the machine, and why the verdict text refuses to call it a fault.

Test it

Do not test it by watching a live feed, because a passing test and a failing test look the same there. Test it by breaking the chain deliberately:

1. Open the panel on any database and note the age of the newest bar.
2. Start Tools -> Bar Replay, set a start date well in the past, and press Play. The panel should announce that replay is active and report the playback position.
3. Press Stop. The replay line should disappear and the newest-bar age should jump back.
4. If you have a feed, use the plugin status context menu to Shutdown (Disconnect), and watch the plugin age climb while the chart itself sits there looking healthy. Reconnect afterwards.

Step four is the whole point of the panel, and step two is how a reader with no feed gets the same result.

Common errors

• The newest-bar age is wrong by a whole number of hours. The database timeshift is not zero, so bar timestamps and the PC clock are on different clocks. The panel prints the timeshift for exactly this reason. Part 20 deals with time zones properly.
• “Current bar completes in” never appears. Status( "lastbartimeleft" ) is documented for time-based bars only and needs a correct timeshift. On tick or volume bars it has nothing to say.
• The panel does not refresh. Either the pane is on an inactive sheet, or the main window is minimised and onlyvisible is at its default.
• A plugin update timestamp that looks impossible. Some sources are documented as sending odd date stamps outside trading hours — the guide names weekends specifically, and notes that the IQFeed plugin sends update stamps only inside regular trading hours. The plausibility band in the formula is there to keep that out of the verdict.

Extension

Add a fourth clock: Status( "lastbarend" ), the DateTime of the end of the last bar. Comparing it with the PC clock tells you whether the bar you are watching should have finished by now — a different question from whether data is arriving, and a useful one when a vendor’s bars stop arriving mid-session.

Doing this lesson without a feed

▷Exercise

Level A path. Everything above is observable on an end-of-day database.

1. Apply data-path-report.afl to a pane on any daily chart. Read the three clocks. The newest bar being a day old is the correct answer, and the panel says why.
2. Run Tools -> Bar Replay and step through a week. Watch the newest-bar age change as the replay position moves, and note that the plugin line still says “not seen” — replay is not a feed, it is a truncation of the database.
3. Write down, for your own setup, which of the seven links in the diagram you actually have. Most readers at this point have links six and seven only, with AmiQuote standing in for links three to five once a day. That is a complete and honest answer, and it is the starting point for the access-levels lesson two pages from now.

Recap

The number on your screen is the output of a chain: venue, venue feed, vendor or broker, the vendor’s local client program, the plugin, the database, and your formula. AmiBroker owns the last three. It buys, operates and warrants nothing above them, which is why so many real-time questions are answered by a subscription page rather than a preference dialog.

Delay accumulates at every hop, and two of the accumulations are settings rather than physics: the base interval at which incoming updates are compressed into bars, and the chart refresh interval that decides how often your formula gets to look. Failures at different links look nearly identical from the bottom of the chain, which is why the only reliable diagnostic measures something other than price — the age of the newest bar, the age of the last plugin update, and the status light.

The next lesson takes the third link seriously and asks what is actually being sold when somebody sells you data.

Check your understanding

Question 1. Your intraday chart has not printed a new bar for twenty minutes. Which single observation best distinguishes a stopped feed from a quiet market?

• The shape of the last few candles
• Whether other symbols on the same venue are still updating
• Whether the volume on the last bar is unusually low
• Whether your backtest of the same symbol still runs

Show the answer and why

Answer: Whether other symbols on the same venue are still updating

A quiet instrument stops printing while the rest of the venue carries on; a stopped feed stops everything downstream of the break. Comparing one symbol with another on the same venue separates the two in seconds. Candle shape and volume are properties of the data you already have, and a backtest reads history, so neither tells you anything about the live path.

Question 2. Which of these are decided by your data vendor rather than by AmiBroker? Select all that apply.

• How much intraday history you can backfill
• Which quote fields carry data at all
• How many bars the database keeps in memory per symbol
• What a given instrument is called as a ticker symbol

Show the answer and why

Answer: How much intraday history you can backfill, Which quote fields carry data at all, What a given instrument is called as a ticker symbol

History depth, field availability and symbology are the vendor’s, expressed through the plugin. Number of bars to load is yours: it is a Database Settings value, and it can only ever keep less than the vendor offers, never more.

Question 3. A formula calls Status("lastrtupdate") on a database with no real-time plugin attached. What does the official documentation say the call returns in that case?

• Zero
• Null
• The timestamp of the last bar
• Nothing — the case is not documented

Show the answer and why

Answer: Nothing — the case is not documented

The Status page documents the code as the date/time of the last update sent by a real-time plugin, and says nothing about the no-plugin case. That is why the example formula validates the value against a plausible band instead of trusting a specific sentinel. Assuming a return value that no page states is exactly the kind of guess that produces confident, wrong code.

Question 4. You add an alert pane to a chart sheet you keep in the workspace but rarely open, and set RequestTimedRefresh with onlyvisible set to False so it keeps running. What happens?

• It refreshes normally, because onlyvisible is False
• It refreshes only while the main window is minimised
• It does not refresh, because panes on inactive sheets are not created until shown
• It refreshes at half the requested rate

Show the answer and why

Answer: It does not refresh, because panes on inactive sheets are not created until shown

The onlyvisible argument covers minimised and off-screen windows. Chart panes on inactive sheets are a separate case: the documentation states they do not really exist until they are shown, so there is nothing to refresh. An alert built this way never fires and looks entirely correct while never firing.

Check my answersClear