Symbols, Watchlists and Categories

The symbol tree on the left of the AmiBroker window looks like Windows Explorer, and the User’s Guide opens its chapter on the subject by telling you that it is not. That warning is there because the resemblance is misleading in a way that costs people symbols.

Here is the difference, in one sentence. In Explorer a file lives in one folder. In AmiBroker a symbol appears in every category leaf it belongs to, while existing exactly once. Apple appears under the Stocks group, under the Nasdaq market, under the Information sector, under a computer-manufacturing industry, and under any watch lists and favourites you put it in — five or six places in the tree, one symbol in the database.

The consequence follows immediately, and it is the reason this lesson opens with a warning rather than with a walkthrough.

⚠Warning

Deleting a symbol from the tree deletes the symbol, not its membership of the category you were looking at. It disappears from every leaf at once, and symbol removal cannot be undone. To remove a symbol from one watch list, use the watch list Remove command instead.

The Symbols window

Window -> Symbols. Three parts: a search box, the category tree, and the symbol list.

The search box does a full-text search with wildcards, against both the symbol and the full name, within the currently selected category. That last clause is the useful bit: select the Technology sector and type A* and you get technology symbols beginning with A. Type *-A0-FX and you get every symbol ending in that substring.

The symbol list sorts by clicking the header row, and the sort order you choose persists across category changes and searches. Columns can be dragged to reorder, so you can put Full Name first if that is how you think.

The symbol you select here is global to the program. Every symbol-aware window and function follows it. That is worth remembering when a chart appears to be showing the wrong instrument: it is showing the globally selected one, unless it has been symbol-locked or put into a link group.

Two kinds of membership

AmiBroker’s categories come in exactly two flavours, and mixing them up is where the design decisions in this lesson come from.

The category system

1. Mutually exclusive: market, group, sector, industry, GICSExactly one of each, always. You can move a symbol between them but you cannot remove the assignment
2. Free membership: watch lists, favourites, indexesZero, one or many. Memberships can be added and removed at will
3. ALLA special category listing every symbol in the database

Mutually exclusive categories are markets, groups, sectors and industries, plus the GICS classification. Every symbol belongs to exactly one of each, whether or not you chose it — a symbol you never classified sits in whatever the defaults were when the database was created. You cannot remove the assignment. The guide’s prescribed workaround is to create an “Unassigned” group, market, sector or industry and move symbols there.

Free membership categories are watch lists, favourites and indexes. A symbol may belong to none, one or many watch lists, and memberships can be removed.

!Important

This distinction determines how you should organise your work. Build your universes as watch lists. A symbol can be in “Course universe”, “Liquid US large caps” and “Currently watching” simultaneously, and you can remove it from any of them without side effects. Try to do the same with groups and you will find yourself fighting the system: a symbol placed in your new group has been taken out of whichever group it was in before.

Use the exclusive categories for what they are for — a single, stable classification of each instrument — and watch lists for everything that changes.

Watch lists

Since version 5.00 AmiBroker supports an unlimited number of watch lists, and one symbol may belong to more than one.

The commands, from Symbol -> Watch List or from the context menu on a watch list in the tree: Add Selected Symbol, Remove Selected Symbol, Type-in Symbols, Import, Export, Erase (make empty), Sort alphabetically, Hide empty watchlists, New watchlist and Delete watchlist. Names are defined in the Symbol -> Categories window, and members appear under the “Watch lists” leaf of the symbol tree.

Several details make watch lists more useful than they first appear.

They are plain text files. Each watch list is a .TLS file, one symbol per line, inside a Watchlists folder within the database, plus an index.txt that defines their order. You can drop your own .tls file into that folder and AmiBroker adds it to index.txt automatically, at the end. Those same files can be opened directly in AmiQuote, which means the list you scan is the list you download.

They preserve insertion order, not alphabetical order. If you sort an Analysis result list by some ranking and then add the results to a watch list, that ranking survives. Use Sort Alphabetically from the context menu when you want alphabetical instead. This is a feature, and any part of the course that ranks symbols relies on it.

Analysis results can populate them directly. Right-clicking a result list offers Add all/selected results to watch list and Replace watch list with the results/selected results. The second empties the list first, which is what you want for a nightly refresh and emphatically not what you want by accident.

Import accepts several formats. .TLS, .CSV and .TXT files are one ticker per line with no other fields. .LST is the Quotes-Plus comma-separated format, of which AmiBroker reads only the first column. Export always produces a plain file, one ticker per line.

Type-in Symbols expects comma-separated symbols, unlike the one-per-line files. And you can add one symbol to several lists at once by Ctrl-clicking list entries in the selector window.

✦Tip

Delete watchlist deletes the list, not the symbols in it. That is the opposite of what deleting from the symbol tree does, and it is the safe operation of the two. If you are ever unsure whether an action removes a membership or an instrument, ask which of the two kinds of category you are operating on.

By default empty watch lists are shown in the tree. Right-click one and choose Hide Empty Watchlists to hide them; select it again to unhide.

Markets and groups

Two mutually exclusive classification axes, independent of sector and industry, named in Symbol -> Categories. Both look like arbitrary labels until you discover that each carries functionality.

Markets carry base indexes. Each market has one or more base indexes attached, used by the Relative Strength indicator, by composites via the composite calculation option, and by Beta. Each market also has a Profile field holding a URL template for the embedded profile browser.

✕Common mistake

Base indexes are attached to markets. That is why relative-strength studies and advance/decline composites break when symbols sit in the wrong market — the calculation is comparing against the base index of a market the symbol should not be in. If a relative-strength figure looks impossible, check the symbol’s market before you check the formula.

Profile URL templates are also per-market, which is why the Profile view is blank for symbols in a market whose template was never filled in.

Groups carry an intraday-settings override. A group can be marked as using its own intraday settings, with its own trading hours, via a button in the Groups tab. This is enormously useful for a database mixing instruments from different sessions — and it is also the thing people forget when exactly one instrument behaves oddly around session boundaries.

Since version 6.10, Move Up and Move Down buttons reorder markets, groups, sectors, industries and watch lists. Use them sparingly: the guide describes reordering as “a non-trivial task as all symbols must be synchronized”, because symbols store the ordinal position of their category rather than its name. Renaming is cheap; reordering is not, and on a large database it takes a while.

Sectors and industries

A two-level hierarchy, and the only one in AmiBroker: industries belong to sectors. Assigning a symbol to an industry automatically implies its sector. In the sample database, General Motors sits in the “Auto & Truck Manufacturers” industry, which places it in the “Consumer/Cyclical” sector without any separate action.

Names are edited in Symbol -> Categories, in the last two tabs. Select an entry, press Enter or the Edit name button, edit, press Enter again. On the Industries tab a Sector combo assigns the industry to its sector.

Where the data lives explains most of the behaviour. The names are stored in broker.workspace, per database. The symbol files store only an ID — an ordinal position. That is why renaming a sector is instant and reordering is expensive, and why two databases can have entirely different taxonomies.

At database-creation time only, AmiBroker seeds the structure from two plain-text template files, broker.sectors and broker.industries, in the main AmiBroker directory. The first is a list of sector names, one per line. The second is a list of industry names, each prefixed by the zero-based index of its sector — 8 Advertising means Advertising belongs to the ninth line of broker.sectors. Once a database exists, those templates are never consulted again. To force an existing database to re-read them you have to delete broker.workspace first, which also destroys that database’s settings and category names.

∴Research note

How many sectors can a database have? The official pages disagree. The categories chapter says “up to 32 sectors and up to 256 industries”. The CategoryAddSymbol function reference documents the sector number as 0 to 63, and the ASCII importer’s $CLEANSECTORS entry describes the wiped structure as Sector 0 through 63. A third page describes fallback naming up to 256.

Treat 256 industries as solid, and the sector ceiling as “at least 32, documented inconsistently”. This course will not assert a number the documentation does not agree on. The practical figure that matters is that markets, groups and industries take numbers 0 to 255 in AFL and sectors 0 to 63, which is the range CategoryAddSymbol and CategoryGetSymbols accept.

A neat trick from the guide, for anyone who wants sectors and does not care about industries: define a one-to-one broker.industries in which each line is the corresponding sector name prefixed with its own index. Setting the industry then becomes equivalent to setting the sector.

Favourites and indexes

Two simple flags, each creating its own leaf in the symbol tree, set in Symbol -> Information or from the tree’s context menu. They behave like free-membership categories, but each is a single Boolean rather than an unlimited set: there is one Favourites leaf and one Indexes leaf.

✕Common mistake

AmiBroker’s own documentation spells this feature both ways — the symbol information window writes “Favourites” and the basic-usage chapter writes “Favorites”. They are the same feature. This is worth knowing generally: when searching AmiBroker’s documentation or its forum, try both spellings.

Assigning categories in bulk

For one symbol, Symbol -> Information — or Window -> Symbol Information — has combos for Market, Group and Industry, along with the Favourites and Index checkboxes and a long list of per-symbol properties. Three of those properties matter later and are worth noting now:

• Continuous quotations must be on to get candlestick charts and manual OHLC entry; otherwise the symbol is treated as price-fixing. This is why $CONT 1 exists in the ASCII importer.
• Round lot size = 0 means “use the global default”; if that is also zero, fractional shares are allowed in the backtester.
• Tick size affects only trades exited by built-in stops and ApplyStop(). The backtester assumes your price arrays already respect tick size and never modifies them.

For many symbols, Symbol -> Organize assignments opens the Assignment organizer: mark a group of symbols and move them between categories in one operation.

⚠Warning

The Assignment organizer is also the only convenient way to delete many symbols at once, via its Delete button — and symbol removal cannot be undone. There is no confirmation-and-recovery path. Back up the database folder before a bulk delete, every time.

The fastest route of all, if you have the data as a file, is the ASCII importer. Since version 5.60 the SECTORNAME and INDUSTRYNAME fields build the whole structure in one pass. The guide’s own recipe:

$FORMAT Ticker,FullName,SectorName,IndustryName
$SEPARATOR ,
$AUTOADD 1
$NOQUOTES 1
$OVERWRITE 1
$CLEANSECTORS 1
$SORTSECTORS 1

$NOQUOTES 1 switches off price validation so a file with no prices imports; $OVERWRITE 1 updates existing symbols as well as new ones. Note $CLEANSECTORS 1, which wipes the existing structure first — appropriate when building a taxonomy from scratch, destructive otherwise.

The same mechanism sits behind Tools -> Update US symbol list and categories, which downloads NYSE, Nasdaq and AMEX symbols with their sectors and industries from amibroker.com. The guide prints a caveat in capitals: that tool wipes any existing sectors and industries.

Using categories from AFL

Everything above becomes useful when a formula can read it. The functions divide into membership tests and lookups.

Fragment — not a complete formula

// Membership tests - true or false for the symbol being analysed
InWatchList( 3 );                     // by ordinal number
InWatchListName( "Course universe" ); // by name, from AmiBroker 5.00

// Lookups - the name of the category this symbol belongs to
MarketID( 1 );      // 0 or omitted gives the number, 1 gives the name
SectorID( 1 );
IndustryID( 1 );
GroupID( 1 );

// The membership list, as a comma-separated string
CategoryGetSymbols( categoryWatchlist, 3 );
CategoryGetName( categoryWatchlist, 3 );

InWatchListName() is generally the better of the two membership tests, because watch list numbers are ordinal positions that move when lists are reordered, while names do not. The trade-off is that a renamed list breaks a name-based test loudly, whereas a reordered list breaks a number-based test silently — and a silent break in a universe filter is a study run over the wrong symbols.

CategoryGetSymbols() returns a comma-separated string, which you split with StrExtract(). It accepts categoryMarket, categoryGroup, categorySector, categoryIndustry, categoryWatchlist, categoryFavorite, categoryIndex, categoryGICS, categoryICB and categoryAll. Parts 15 and 16 use it heavily; the documentation warns that on a large watch list it can be slow, and that such code should be avoided in real-time mode.

Auditing your universe

Before trusting any of this, look at it.

Complete runnable AFL

universe-audit.afl

// ===========================================================================
//  Universe audit
//  Prints how every symbol in the database is filed: which market, group,
//  sector and industry it belongs to, and whether it is a member of one chosen
//  watch list. Run it before you trust any Analysis filter, because a filter
//  that matches nothing and a filter that is spelled wrongly look identical
//  from the outside - both return an empty result.
//
//  HOW TO RUN
//    Analysis window: Apply to = All symbols, Range = 1 recent bar(s).
//    Category membership does not change bar by bar, so one bar is enough and
//    the exploration finishes far faster.
//    Set the watch list number in the Analysis window Parameters dialog.
//
//  ASSUMPTIONS, STATED SO THEY CAN BE CHECKED
//    - Market, group, sector and industry are mutually exclusive in AmiBroker:
//      every symbol has exactly one of each, whether or not you chose it.
//      A symbol you never classified sits in whatever the defaults were.
//    - Watch list membership is free: a symbol may be in none, one or many.
//      This formula reports one watch list at a time, by number.
//    - Watch list numbers are ordinal positions, not names. If you reorder
//      watch lists, the numbers move with them. CategoryGetName() is printed
//      alongside so you can confirm you are testing the list you meant.
// ===========================================================================

WatchListNumber = Param( "Watch list number", 0, 0, 63, 1 );

Filter = Status( "lastbarinrange" );
SetOption( "NoDefaultColumns", True );

WatchListName = CategoryGetName( categoryWatchlist, WatchListNumber );
IsMember      = InWatchList( WatchListNumber );

AddTextColumn( Name(),          "Symbol",    1.0, colorDefault, colorDefault,  80 );
AddTextColumn( FullName(),      "Full name", 1.0, colorDefault, colorDefault, 200 );
AddTextColumn( MarketID( 1 ),   "Market",    1.0, colorDefault, colorDefault, 120 );
AddTextColumn( GroupID( 1 ),    "Group",     1.0, colorDefault, colorDefault, 120 );
AddTextColumn( SectorID( 1 ),   "Sector",    1.0, colorDefault, colorDefault, 140 );
AddTextColumn( IndustryID( 1 ), "Industry",  1.0, colorDefault, colorDefault, 180 );
AddTextColumn( WatchListName,   "Watch list tested", 1.0, colorDefault, colorDefault, 140 );

// Printed as words rather than as 1 and 0, so the table is readable without a
// legend and survives being pasted into a spreadsheet.
AddTextColumn( WriteIf( IsMember, "member", "-" ), "Member?", 1.0,
               colorDefault, colorDefault, 80 );

AddColumn( Close, "Last close", 1.2 );

// Group the report by sector, then by industry inside each sector, which is
// the order in which the category tree itself is drawn.
SetSortColumns( 5, 6 );

Download universe-audit.afl51 lines

Run it as an exploration with Apply to = All symbols and Range = 1 recent bar(s) — category membership does not vary bar by bar, so one bar per symbol is enough and the run finishes far faster. Set the watch list number in the Analysis window’s Parameters dialog.

=Expected result

One row per symbol, showing its market, group, sector and industry by name, the name of the watch list you are testing, and whether the symbol is a member of it. Sorted by sector and then industry, so the taxonomy reads as a taxonomy.

On the sample DJIA database, every symbol should have a real sector and industry name because the sample database ships classified. On a database you built by importing a CSV with no category fields, expect every symbol to show whatever the default names were — and that is exactly the gap you want to see before writing a filter that depends on them.

✕Common mistake

A filter that matches nothing and a filter that names a category wrongly produce identical output: an empty result. This exploration is how you tell them apart. Run it first, confirm the categories are what you think, then write the filter.

A universe you can actually use

Concretely, for the rest of this course:

1. Create a watch list called Course universe and put the symbols you intend to study in it. Keep it to a size you can inspect by eye — a few dozen is plenty for learning, and Part 12 explains how and when to scale up.
2. Create a second watch list called Data problems, and move anything the import health check flagged into it rather than deleting it. Symbols you excluded for a reason are evidence; symbols you deleted are gone.
3. Leave markets, groups, sectors and industries as they arrived, unless you have a real classification to import. A partly filled taxonomy is worse than an empty one, because it looks trustworthy.
4. Write the watch list names into your setup record. Names, not numbers.

What changed in your understanding

You now know that AmiBroker’s symbol tree shows memberships rather than locations, and what follows from that: a symbol in six leaves is one symbol, and deleting it from one leaf deletes it everywhere.

You know the two membership models and which to use for what — exclusive categories for a single stable classification, watch lists for everything that changes — and you know why that is not merely a preference: exclusive categories cannot have their assignments removed at all.

You know that watch lists are text files inside the database, that they preserve insertion order, and that Analysis results can be poured straight into them. And you know how to read all of it from AFL, and how to check it before you rely on it.

Check your understanding

Question 1. You right-click a symbol under the Technology sector leaf and choose Delete. What happens?

• The symbol is removed from the Technology sector only
• The symbol is deleted from the database entirely and cannot be recovered
• The symbol is moved to an Unassigned sector
• Nothing — sector membership cannot be edited from the tree

Show the answer and why

Answer: The symbol is deleted from the database entirely and cannot be recovered

The tree shows memberships, not locations, so there is no "remove from this leaf" meaning available. Deleting removes the symbol itself, from every category at once, and symbol removal cannot be undone. Sector membership is mutually exclusive and cannot be removed at all — the documented workaround is an Unassigned sector you move symbols into.

Question 2. Which of these can a single symbol belong to more than one of at the same time? Select all that apply.

• Watch lists
• Markets
• Sectors
• Groups

Show the answer and why

Answer: Watch lists

Watch lists, favourites and indexes are free-membership: zero, one or many. Markets, groups, sectors and industries are mutually exclusive — exactly one of each, always, whether or not you chose it. That asymmetry is the single most useful thing to know about the category system.

Question 3. Why does this course recommend InWatchListName() over InWatchList() for universe filters?

Filter = InWatchListName( "Course universe" );

• It is faster on large databases
• Watch list numbers are ordinal positions that shift if lists are reordered
• InWatchList only works in explorations
• InWatchList is deprecated

Show the answer and why

Answer: Watch list numbers are ordinal positions that shift if lists are reordered

Watch list numbers are positions in an ordered list, and reordering the lists moves them. A number-based filter then silently runs over a different universe, which is far harder to notice than a name-based filter breaking loudly after a rename. Prefer the failure you can see.

Question 4. Your exploration filtered on a sector returns no rows. What should you check first?

• Whether the formula has a syntax error
• Whether the symbols actually carry the sector assignment you assume
• Whether the Analysis window has enough threads
• Whether the database needs re-indexing

Show the answer and why

Answer: Whether the symbols actually carry the sector assignment you assume

An empty result is ambiguous by construction: it looks the same whether nothing matched or nothing was ever classified. Symbols imported from a CSV with no category fields keep whatever default names the database was created with, so a sector filter over them matches nothing. Audit the categories, then write the filter.

Check my answersClear