Survivorship, Delistings and Index Membership

Every defect in the previous lesson was a problem with a number inside a series. This one is different in kind: the numbers can all be perfect and the answer still wrong, because the wrong set of series is present. It is the most under-appreciated failure in retail-level trading research, it is easy to describe and hard to fix, and Part 30 returns to it when you have a backtester in front of you. This lesson establishes exactly what it is, how large it can be, and what remains honest to claim when you cannot remove it.

The bias, stated precisely

What you compute, and what you meant to compute

Suppose you download the current constituents of a large market index, load twenty years of history for each, and test a rule. What you have computed is:

the average outcome of the rule over the past twenty years, conditional on the instrument still being a member of that index today.

What you almost certainly meant to compute is:

the average outcome of the rule over the past twenty years, using only information that was available at the time each decision was made.

Those are different quantities. The first conditions on an event that had not happened yet at the moment of every trade in the test. In the language of research design, it is selection on the outcome: the criterion for entering the sample is correlated with the thing being measured.

Why the correlation is structural rather than accidental

Membership of a database of currently listed instruments is not random. An instrument is in it because it did not fail, was not acquired, was not taken private, and was not removed for falling below a listing standard. Every one of those exclusions is related to performance. That is the whole mechanism — there is nothing subtle in it, which is why the bias is so reliable and so easy to overlook.

!Important

Survivorship bias is not a data defect that better cleaning removes. Your database is telling the exact truth about every symbol it contains. The error is in the inference, not in the data — which is why no check on individual series can ever find it.

Where the missing instruments went

“Delisted” is a single word covering several outcomes with different consequences.

Outcome	Typical terminal value	Effect on a study that omits it
Bankruptcy or liquidation	Near zero	Removes the worst outcomes entirely
Removed for failing listing standards	Low, often continues trading elsewhere	Removes a poor outcome, and the later history
Acquired for cash	Usually a premium to the last price	Removes a good outcome
Merged for shares	Continues inside another entity	Removes a continuing position
Taken private	Cash at a negotiated price	Removes a completed outcome
Ticker changed	Unchanged	May duplicate or split one company’s history

The first two remove bad outcomes; the third and fifth remove good ones. They do not cancel. In most equity databases the dominant term is the removal of failures, because failures are more numerous than acquisitions in a broad universe and because the size of the loss in a failure is far larger than the size of a typical acquisition premium. The result is that measured performance is typically biased upward — but the direction is an empirical property of your market and your period, not a law, and the right response is to measure it rather than to assume it.

There is a second, less discussed consequence. Survivorship does not only flatter returns; it flatters risk. Maximum drawdown, worst trade, win rate and the length of losing streaks all improve when the instruments that went to zero are absent. Those are precisely the numbers people look at for reassurance.

How much can it move a result?

The following arithmetic is entirely synthetic. Its purpose is to establish an order of magnitude, not to describe any real market.

Setup. A universe of 100 hypothetical instruments, equally weighted, held for ten years. Ninety of them double, ending at 2.00 for every 1.00 invested. Ten fail completely and end at 0.00.

Measured on the survivors only — which is what a database of currently listed instruments contains — average terminal wealth is 2.00. That is +100 per cent over ten years, or about 7.2 per cent a year.

Measured on the complete universe, average terminal wealth is (90 × 2.00 + 10 × 0.00) / 100 = 1.80. That is +80 per cent over ten years, or about 6.1 per cent a year.

A ten per cent failure rate over a decade — unremarkable for a broad equity universe — moves the annualised figure by roughly 1.1 percentage points. That is already larger than the entire edge many published strategies claim.

And a rule can make it far worse

The example above assumes the strategy holds everything. Real rules select, and the selection interacts with the bias.

Consider a rule that deliberately buys the most beaten-down names — a value screen, a mean-reversion rule, a “falling knife” entry. In reality, a substantial share of its picks would come from exactly the population that later failed. Suppose the rule holds ten positions, and in the complete universe five of them are among the ten that went to zero:

• Complete universe: (5 × 2.00 + 5 × 0.00) / 10 = 1.00. Ten years, zero return.
• Survivor-only database: the five failures are not present, so the rule picks five other names, all of which survive and double. Terminal wealth 2.00. Ten years, +100 per cent.

The bias here is not 1.1 percentage points a year. It is the entire result. The strategy that looks best in a survivorship-biased database is often the one whose bias is largest, because both are driven by the same thing: exposure to instruments that were in trouble.

∴Research note

This is why survivorship bias cannot be handled with a fixed haircut. “Subtract one per cent a year and it still works” is not a defence, because the size of the bias depends on the rule. A trend-following rule that only holds instruments making new highs is much less exposed than a distressed-value rule, and the two need different treatment.

Historical index membership: a second problem, stacked on the first

Index membership introduces a bias that is independent of, and often larger than, plain survivorship.

An index is a time-varying set. Companies are added — usually after they have already grown large and performed well — and removed after they have shrunk or failed. Testing “the current members of the index, over twenty years of history” therefore does two things at once: it drops the companies that were removed, and it back-projects today’s membership on to a past in which nobody could have known it.

Index membership is a set that changes (synthetic companies)

Company A

in the index throughout

Company B

not a memberadded after a large rise

Company C

memberremoved after a collapse

Today’s list

A and B only — C is invisible

2016201920222025

A study built on today’s list holds Company B for the five years before it was eligible, and never holds Company C at all. The first is a look-ahead error; the second is survivorship. They are separate faults with separate mechanisms, and a study can easily have both.

✕Common mistake

AmiBroker’s watch lists describe the present, not history. InWatchList() returns a NUMBER, not an array — one value for the whole symbol, with no bar-by-bar dimension. A filter written as Filter = InWatchListName( "Index members" ); therefore applies today’s membership to every bar in the test, including bars from years before the symbol joined. This is not a bug in AmiBroker; watch lists were never intended as historical records. It is a trap only if you expect them to be something they are not.

Ticker changes and ticker reuse

A ticker is a label on a listing, not an identity. Two failure modes follow.

A company changes ticker. Its history is now split across two symbols, and neither one is complete. AmiBroker provides Symbol → Merge for exactly this case: select the new ticker, choose the old one to merge with, and optionally delete the old ticker and copy it into the new symbol’s alias field. Doing that repairs the series — and destroys the record that the ticker changed, unless you keep the alias.

A ticker is reused. An exchange reassigns a symbol freed by a delisting to an unrelated new listing. If your database appends the new company’s data to the old company’s file, you now hold a single continuous series covering two different businesses, with an invented gap or jump where they join. Nothing about that series looks wrong. Every check in the previous lesson passes.

The only defence is provenance: a dataset that identifies companies by a stable identifier rather than by ticker, and records the mapping of tickers to companies over time.

What a point-in-time universe actually requires

“Point-in-time” means: a query for date D returns exactly what was knowable on date D. For a tradable universe, that requires the dataset to record, at minimum:

1. Listing date for every instrument, so nothing is held before it existed.
2. Delisting date and reason, so instruments leave when they left.
3. Terminal value or consideration — cash, shares, or zero. Without it, a delisting is silently treated as neutral, which is wrong in both directions.
4. Index or universe membership intervals, with add and remove dates, so membership can be evaluated as of a bar rather than as of today.
5. Ticker history, mapping symbol to entity over time, so reuse and renaming do not splice unrelated series together.
6. Corporate-action history keyed to the entity, not to the ticker.
7. As-of semantics for restatements: values later revised must not silently replace what was reported at the time.

Miss any one and the universe cannot be reconstructed as it was. Most retail data sources provide none of them: they provide current symbols and their price history, which is a different product.

AmiBroker itself is agnostic here. Its database will hold delisted symbols perfectly well — a symbol whose last quote is from 2011 charts and computes exactly like any other. What AmiBroker does not do is supply them, or record when a symbol belonged to a list. Both must come from your data source. The official data-source listing does identify vendors whose offering explicitly includes delisted symbols, and Part 18 goes into the specifics; the point for now is that this is a purchasing decision made before you build the database, not a setting you can change afterwards.

Diagnosing your own database

You can find out where you stand in about a minute, and the answer is usually sobering.

Goal

Produce one row per symbol showing when its history starts, when it stops, and how long ago it last traded — so you can count how many instruments in your database stopped existing.

Complete formula

Complete runnable AFL

series-integrity-report.afl

// series-integrity-report.afl
// Part 2 - Survivorship, Delistings and Index Membership
//
// One row per symbol: when its history starts, when it stops, how many bars it
// holds, and how long ago it last traded.
//
// The question this answers is not "is this symbol any good". It is: does this
// database contain the instruments that stopped existing? A database in which
// every symbol traded within the last few days holds no dead companies at all,
// and no historical study run over it can see failure.
//
// Assumptions:
//   - Analysis window, Exploration mode, Apply to: All symbols,
//     Range: All quotations.
//   - Daily bars. On intraday data the day counts are still right, but bar
//     counts are not comparable between symbols with different histories.
//   - "Days since last bar" is measured against your computer's clock, so a
//     database you have not updated recently will report every symbol as stale.
//     Update first, then run this.

_SECTION_BEGIN("Series integrity report");

StaleDays = 30;   // a symbol quiet for longer than this needs an explanation

FirstBar = Status( "firstbarinrange" );
LastBar  = Status( "lastbarinrange" );

FirstStamp = ValueWhen( FirstBar, DateTime() );
LastStamp  = DateTime();

// DateTimeDiff returns a difference in seconds. There are 86400 in a day.
DaysSinceLastBar = DateTimeDiff( Now( 5 ), LastStamp ) / 86400;

BarsHeld      = Cum( 1 );
BarsWithTrade = Cum( Volume > 0 );
BarsNoTrade   = BarsHeld - BarsWithTrade;

// The verdict is written as text, not as a colour, so that it survives being
// copied, sorted, printed or read aloud.
Verdict = WriteIf( LastValue( DaysSinceLastBar ) > StaleDays,
                   "STOPPED - explain why",
                   "current" );

Filter = LastBar;

AddColumn(     FirstStamp,       "First bar",           formatDateTime );
AddColumn(     LastStamp,        "Last bar",            formatDateTime );
AddColumn(     BarsHeld,         "Bars",                1.0 );
AddColumn(     BarsNoTrade,      "Bars with no trade",  1.0 );
AddColumn(     DaysSinceLastBar, "Days since last bar", 1.0 );
AddTextColumn( Verdict,          "Status" );
AddColumn(     Close,            "Last close",          1.4 );

_SECTION_END();

Download series-integrity-report.afl54 lines

✦Tip

Another Exploration to keep for later — it needs a database (Part 3) and the Analysis window (Part 12). What matters now is the reasoning behind it, which needs no software at all.

How it works

Status( "firstbarinrange" ) and Status( "lastbarinrange" ) mark the ends of each symbol’s history. ValueWhen() captures the timestamp at the first of those. Cum( 1 ) counts bars, and Cum( Volume > 0 ) counts the bars on which anything traded, so their difference is the number of bars with no trade. DateTimeDiff( Now( 5 ), … ) returns seconds between the current clock and the symbol’s last bar, which divided by 86,400 gives days. Reporting only on the last bar of each symbol collapses all of it to one row.

Key functions

• Now( 5 ) returns the current date and time as an encoded value, matching what DateTime() produces for a bar.
• DateTimeDiff( a, b ) returns the difference between two encoded date/time values, in seconds.
• WriteIf( condition, trueText, falseText ) picks one of two strings; passing LastValue() of an array makes the condition a single value, which is what a text column needs.
• AddTextColumn( string, name ) adds a column of text rather than numbers.

Expected result

A table you can sort by “Days since last bar”. The number to look at is how many rows read STOPPED.

⌀Data quality warning

If a database claiming twenty years of a broad equity market contains no symbols whose history ends years ago, it holds no delisted companies at all. Every historical study run over it is survivorship-biased, by construction, regardless of how carefully the rest of the work is done. That single count tells you more about the trustworthiness of your backtests than any statistic the backtester will produce.

Test it

Pick one symbol the report marks as stopped and check what happened to it: search the ticker and the last-quote date. You should be able to identify a real event — a merger, a bankruptcy, a delisting, a ticker change. If you cannot find any event, the more likely explanation is a failed download, which is a different problem worth knowing about too.

Common errors

• Running it against a stale database, so every symbol appears stopped. Update first.
• Running it on a database fed by a plugin whose symbol list is maintained automatically: some plugins remove symbols when a subscription lapses, which changes the answer without changing anything about the market.
• Reading a late first bar as a defect. A company that listed in 2018 has no history before 2018, and that is correct. Database Purify’s own documentation warns against treating a shorter history as a problem for this reason.

Extension

Add a column counting bars in the last 250 sessions, and compare it with the total bar count. A symbol that traded heavily for a decade and has thirty bars in the last year is usually being wound down, and is a candidate for exclusion on liquidity grounds well before it formally delists. The related Tools → Cleanup database script takes the same idea further: it scans for symbols whose latest quote is old enough to suggest they are no longer traded, and offers to remove them — which is a useful housekeeping tool and, applied carelessly, an excellent way to manufacture survivorship bias in a database that did not previously have it.

What to do when you cannot fix it

Point-in-time data costs money, and this course is designed to be completable without buying any. So the honest question is what remains legitimate without it. In order of usefulness:

1. Say so, every time. Every result you record should carry its universe construction: how the symbol list was obtained, on what date, and whether it includes delisted instruments. A result without that provenance is not interpretable later, including by you.
2. Bound the bias rather than ignore it. Estimate an annual attrition rate for your market, apply the arithmetic from earlier in this lesson, and ask whether the measured edge survives an adjustment of that size. If it does not, you have your answer without spending anything.
3. Prefer questions the bias cannot reach. A study of a single continuous series — a broad index, an index future, a currency pair — involves no cross-sectional universe selection, so cross-sectional survivorship simply does not arise. A great deal of useful research about indicators, regimes and volatility can be done this way.
4. Avoid the rules where the bias is largest. Distressed value, deep mean reversion, micro-cap screens and anything that systematically buys weakness are the most exposed. Trend-following rules that hold only instruments making new highs are the least exposed — less exposed, not immune.
5. Never compare a biased strategy result with an unbiased benchmark. A published index series records what the index actually did at the time, including the constituents that were later removed. Measuring your survivor-only strategy against it puts a bias on one side of the comparison only, and flatters the strategy twice.
6. Narrow the claim to what you tested. “This rule, applied to these 100 companies that are listed today, over this period” is a defensible and genuinely interesting statement. It is simply a much smaller statement than “this rule works”, and writing it out in full is usually enough to prevent the overreach.

?Reality check

A claim you will meet often: “survivorship bias is a rounding error for large-cap strategies, because big companies rarely disappear.” The claim is testable rather than obviously false, and testing it is instructive. Ask what fraction of the index members of twenty years ago are still members, what happened to the rest, and whether the strategy in question would have held them on the way down. On the evidence available to a retail researcher, the honest conclusion is usually that the effect is smaller for large caps than for small ones and still large enough to matter — which is not the same as a rounding error.

Recap

Survivorship bias is a selection problem, not a data-quality problem: the series in your database can be flawless while the set of series is wrong. It arises because presence in a current symbol list is conditional on outcomes that had not occurred at the time of the trades being simulated. Index membership adds a second, independent fault, because membership itself is awarded after good performance, and AmiBroker’s watch lists — being current sets rather than dated ones — apply today’s membership to all of history. A synthetic ten per cent failure rate over a decade moves an annualised return by roughly a percentage point, and a rule that selects for weakness can have its entire result manufactured by the bias. Point-in-time data fixes it and costs money; in its absence, the defensible course is to measure what you can, bound what you cannot, prefer questions the bias cannot reach, and state the universe construction alongside every result.

The lab that follows makes the general argument of this part concrete: you will damage a series yourself, run one rule on both copies, and measure what the damage did.

Check your understanding

Question 1. You download today’s constituents of a major index and backtest a rule over twenty years of their history. Which biases are present?

• Survivorship only
• Look-ahead on index membership only
• Both survivorship and look-ahead on membership
• Neither, provided the price data is adjusted correctly

Show the answer and why

Answer: Both survivorship and look-ahead on membership

Companies removed from the index during the period are absent, which is survivorship. Companies added during the period are held from before they qualified, which is look-ahead on membership. The two are independent faults and this design has both.

Question 2. A hypothetical universe of 100 instruments is held for ten years. Ninety double and ten go to zero. What does measuring only the survivors do to the annualised return?

• Nothing, because the failures contributed no return
• Raises it from about 6.1 per cent to about 7.2 per cent a year
• Lowers it, because the surviving sample is smaller
• Raises it by exactly ten per cent, matching the failure rate

Show the answer and why

Answer: Raises it from about 6.1 per cent to about 7.2 per cent a year

Complete terminal wealth is 1.80 per unit invested, about 6.1 per cent a year; survivors alone end at 2.00, about 7.2 per cent. The gap is roughly 1.1 percentage points a year from a ten per cent failure rate — larger than many claimed edges.

Question 3. Why can no per-symbol data-quality check detect survivorship bias?

• Because the check would need real-time data
• Because the bias lives in which symbols are present, not in any symbol’s values
• Because AmiBroker does not store delisting dates
• Because survivorship only affects intraday data

Show the answer and why

Answer: Because the bias lives in which symbols are present, not in any symbol’s values

Every series in a survivorship-biased database can be perfectly accurate. The fault is in the composition of the set, which is invisible from inside any individual series — which is why the diagnostic in this lesson counts symbols rather than inspecting bars.

Question 4. Which of these reduce your exposure to survivorship bias without buying point-in-time data? Select all that apply.

• Studying a single continuous index series instead of a cross-section
• Recording how and when the symbol list was obtained
• Running the backtest over a longer period
• Comparing the strategy against a published index total return

Show the answer and why

Answer: Studying a single continuous index series instead of a cross-section, Recording how and when the symbol list was obtained

A single series involves no universe selection, and provenance keeps the result interpretable. A longer period generally increases attrition and therefore the bias. Comparing against a published index puts an unbiased series on one side of the comparison and a biased one on the other, which flatters the strategy rather than controlling for anything.

Check my answersClear