Challenge: 180% a Year with a 3% Drawdown

The report

Somebody shows you this and asks whether you would like to invest.

Weekly momentum breakout with a tight 3% stop. Full position, one name at a time, so the winner compounds. Tested on 40 small caps from my watchlist, April 2020 to December 2023.

Annual return 180%. Maximum drawdown 3%. 41 trades. 78% winners.

!Important

Those figures are illustrative. They are the numbers used to frame the exercise, not a measurement anyone should reproduce or cite. Running the formula below on your own data will produce different numbers, and they will be just as meaningless. The exercise is the diagnosis, not the arithmetic.

The symptoms

Before you read a line of code, three things in that summary should already be uncomfortable.

A maximum drawdown smaller than the stop distance. The system claims a 3% stop and a 3% maximum drawdown. A single stopped-out trade at full position size costs 3%. So this system claims never to have had two losing trades in a row, and never to have lost more than one stop’s worth across the whole period, while turning over 41 trades.

180% a year, compounded over nearly four years. That is roughly a 50-fold increase. If a strategy of this simplicity did that reliably, the capacity of the instruments involved would be exhausted long before the fourth year, and everyone would be running it.

A universe of “40 small caps from my watchlist”, assembled after the period ended.

The evidence you have

Complete runnable AFL

impossible-backtest.afl

// impossible-backtest.afl
// Part 30 - Challenge: 180% a Year with a 3% Drawdown
//
// ####################################################################
// #  THIS FORMULA IS DELIBERATELY BROKEN. DO NOT TRADE IT, DO NOT     #
// #  COPY IT INTO YOUR OWN WORK, AND DO NOT BELIEVE ANY NUMBER IT     #
// #  PRODUCES. It exists so that you can find out why the numbers     #
// #  are impossible, before someone shows you a report like it and    #
// #  asks for money.                                                  #
// ####################################################################
//
// This is the formula behind the report in the challenge. It was written by
// somebody who was pleased with it. Every line is legal AFL. It compiles, it
// runs, it produces a report, and the report is meaningless.
//
// WHAT ITS AUTHOR SAID ABOUT IT (their words, reproduced, not endorsed):
//   "Weekly momentum breakout with a tight 3% stop. Full position, one name at
//    a time, so the winner compounds. Tested on 40 small caps from my
//    watchlist, April 2020 to December 2023. Annual return 180%, max drawdown
//    3%, 41 trades, 78% winners."
//
// The claimed figures above are ILLUSTRATIVE - they are the numbers used to
// frame the exercise, not a measurement anyone should reproduce or cite. Your
// own run of this file on your own data will produce different figures, and
// they will be just as meaningless.
//
// The author's stated settings, which matter as much as the code:
//   Periodicity            Daily
//   Trade price            Open, for all four of buy/sell/short/cover
//   Delays                 0 / 0 / 0 / 0
//   Allow same bar exit    ON
//   Commissions            none
//   Account margin         100
//   Limit trade size as %
//     of entry bar volume  0 (off)
//   Range                  1 April 2020 to 31 December 2023
//   Universe               "40 small caps from my watchlist", assembled in 2024
//
// Your task is in the lesson. Read the code before you read the solution.

SetOption( "InitialEquity", 100000 );
SetOption( "MaxOpenPositions", 1 );
SetOption( "AllowSameBarExit", True );
SetOption( "ActivateStopsImmediately", True );
SetOption( "CommissionMode", 1 );
SetOption( "CommissionAmount", 0 );

SetTradeDelays( 0, 0, 0, 0 );

BuyPrice   = Open;
SellPrice  = Open;
ShortPrice = Open;
CoverPrice = Open;

PositionSize = -100;

// The parameter values below are the ones that came top of a four-parameter
// grid search run over the same data the results are quoted from.
BreakoutFrac = Optimize( "Breakout fraction of weekly high", 0.985, 0.95, 1.00, 0.005 );
MomPeriod    = Optimize( "Momentum period",                     17,    5,   40, 1 );
MomThreshold = Optimize( "Momentum threshold %",                 6,    1,   20, 1 );
StopPct      = Optimize( "Max loss stop %",                      3,    1,   10, 0.5 );

// ---------------------------------------------------------------------
// Setup: "price is pushing into the top of this week's range while momentum
// is strong".
// ---------------------------------------------------------------------
WeeklyHigh = TimeFrameGetPrice( "H", inWeekly );
WeeklyClose = TimeFrameGetPrice( "C", inWeekly );

Momentum = ROC( Close, MomPeriod );

Setup = Close > BreakoutFrac * WeeklyHigh
    AND Momentum > MomThreshold;

Buy  = Setup;
Sell = Close < MA( Close, 10 );

// When more than one symbol qualifies, prefer the one with the most room left
// to the weekly close.
PositionScore = 100 * ( WeeklyClose - Close ) / Close;

ApplyStop( stopTypeLoss, stopModePercent, StopPct, 1 );

_SECTION_BEGIN( "Impossible backtest view" );
Plot( Close, "Close", colorDefault, styleCandle );
Plot( WeeklyHigh, "Weekly high as the formula sees it", colorRed, styleLine );
PlotShapes( Buy * shapeUpArrow, colorGreen, 0, Low );
PlotShapes( Sell * shapeDownArrow, colorRed, 0, High );
_SECTION_END();

Download impossible-backtest.afl90 lines

Read the header block as carefully as the code. The stated settings — delays of zero, trade price Open, same-bar exit on, no commissions, no trade-size limit, that date range and that universe — are as much a part of the evidence as any line of AFL.

Your task

Working alone, and before scrolling to the hints, write down every reason you can find why this result is not achievable. For each one, note:

• what is wrong
• which line or setting causes it
• which direction it biases the result
• roughly how much of the result you think it accounts for

Aim for at least six. There are more than six.

Then rank them by how much of the 180% you believe each is responsible for. The ranking is the hard part and the part that transfers.

▷Exercise

Do not skip straight to the solution. The value of this exercise is entirely in the attempt — the point is not to know these faults, it is to become the kind of reader who spots them unprompted when the report is not labelled as a challenge.

Hints, in increasing order of directness

Read one at a time and go back to the formula between each.

Hint 1. Two of the faults are about when information became available. One is in the settings, one is in a function call.

Hint 2. Look up TimeFrameGetPrice’s signature and read what its third and fourth arguments default to.

Hint 3. SetTradeDelays( 0, 0, 0, 0 ) with BuyPrice = Open. On which bar is the entry decided, and on which bar is it filled?

Hint 4. Read the fourth argument of ApplyStop. What does the backtest do when a bar gaps straight through a 3% stop?

Hint 5. Look at how PositionScore is computed. Is that quantity known at the moment the ranking has to happen?

Hint 6. The parameters are Optimize() calls. Where did the quoted defaults come from, and what data were they chosen on?

Hint 7. “40 small caps from my watchlist”, assembled in 2024, tested from 2020. Which companies are missing from that list?

Hint 8. MaxOpenPositions is 1 and PositionSize is −100. What does a 3% stop mean for account risk, and what does one gap mean?

The full diagnosis

Ranked by how much of the result each fault is likely to account for.

1. The weekly high is known before the week ends

Fragment — not a complete formula

WeeklyHigh = TimeFrameGetPrice( "H", inWeekly );

The signature is TimeFrameGetPrice( pricefield, interval, shift = 0, mode = expandFirst ), and the documentation is unambiguous about the consequence:

if shift = 0 compressed data may look into the future ( weekly high can be known on monday ). If you want to write a trading system using this function please make sure to reference PAST data by using negative shift value.

With the defaults, Monday’s bar knows the entire week’s high. The condition Close > 0.985 * WeeklyHigh is therefore asking “is today’s close within 1.5% of a level that will not be established until Friday?” — and buying when the answer is yes.

This single fault is capable of producing the entire result on its own. It is not a subtle bias; it is a machine for buying things that are about to go up.

The fix: TimeFrameGetPrice( "H", inWeekly, -1 ) — the previous completed weekly bar.

2. The rank uses a price from the future

Fragment — not a complete formula

PositionScore = 100 * ( WeeklyClose - Close ) / Close;

Same fault, worse consequence. WeeklyClose with shift 0 is the week’s closing price, published on the week’s first bar. The formula ranks candidates by how much further they will rise by Friday, and then takes the highest-ranked one.

It is not selecting well. It is being told the answer.

3. Deciding on the close, filling at the same bar’s open

Fragment — not a complete formula

SetTradeDelays( 0, 0, 0, 0 );
BuyPrice = Open;

The entry condition uses Close. The fill uses that same bar’s Open — a price that printed hours before the close existed.

On any bar the system chooses to buy, it buys at the start of a bar it already knows ended higher. Combined with AllowSameBarExit turned on, complete round trips can happen inside a single bar at prices the rules could only have known after that bar closed.

AllowSameBarExit is not itself a bug — with HoldMinBars at zero it is a documented model for entry-then-exit within one bar. It is a bug here, because of what it is combined with.

The fix: delays of 1 on all four, prices from the next bar’s open, and same-bar exit off.

4. The stop fills at the stop price, always

Fragment — not a complete formula

ApplyStop( stopTypeLoss, stopModePercent, StopPct, 1 );

ExitAtStop = 1 fills at exactly the level you nominated on every triggering bar — including every bar that gapped straight through it.

This is what makes a 3% maximum drawdown arithmetically possible. In this simulation, no loss can ever exceed 3%, because every stop is honoured at the level. In reality a small-cap can gap 20% overnight, and with a full position and one name at a time, that is a 20% account loss.

The fix: ExitAtStop = 2, so a gap is paid at the next bar’s price.

5. Parameters chosen from the same data the result is quoted on

Fragment — not a complete formula

BreakoutFrac = Optimize( "Breakout fraction of weekly high", 0.985, 0.95, 1.00, 0.005 );
MomPeriod    = Optimize( "Momentum period",                     17,    5,   40, 1 );
MomThreshold = Optimize( "Momentum threshold %",                 6,    1,   20, 1 );
StopPct      = Optimize( "Max loss stop %",                      3,    1,   10, 0.5 );

Four parameters. 11 × 36 × 20 × 19 is roughly 150,000 combinations, evaluated against 41 trades of data — and the quoted defaults are the winning cell.

MomPeriod = 17 is the tell. Nobody chooses 17 from theory.

6. Survivorship, twice over

The universe is “40 small caps from my watchlist”, assembled in 2024 and tested from 2020. Every company that failed, was delisted, or was taken over during the period is absent. Small caps are exactly the population where that attrition is largest.

And it is worse than ordinary survivorship, because a watchlist is a curated list. The selection was made by a person who had already seen how those names performed.

7. No costs at all

Commission is zero and there is no slippage in the price arrays. On small caps, with 41 round trips, the spread alone would be a substantial fraction of the gross result.

8. One position, full size, no participation limit

MaxOpenPositions = 1 and PositionSize = -100 means the whole account in one small-cap name at a time, growing with the equity curve. By the end of a 50-fold increase, each position would be enormous relative to the instruments involved — and the trade-size limit is explicitly off.

9. The period was chosen

April 2020 to December 2023 begins at a market low. That is not neutral, and the choice was available only in hindsight.

The repaired formula

Complete runnable AFL

impossible-backtest-repaired.afl

// impossible-backtest-repaired.afl
// Part 30 - Challenge: 180% a Year with a 3% Drawdown (solution formula)
//
// PURPOSE
//   The same trading idea as impossible-backtest.afl, rebuilt so that every
//   number it produces could in principle have been earned. It is not a better
//   system. It is the same system with the fiction removed, and the honest
//   version of a fictional result is usually a much smaller one - sometimes a
//   negative one. That is the point of the exercise.
//
// WHAT CHANGED, AND WHY
//   1. Signals are decided on the CLOSE and filled on the NEXT bar's OPEN.
//      SetTradeDelays(1,1,1,1) and Open-based price arrays. The broken version
//      decided on the close and filled at the same bar's open.
//   2. "Allow same bar exit" is off. It is not a bug by itself - with
//      HoldMinBars at zero it simply means entry-then-exit within one bar, and
//      that is a legitimate model for some designs. It is a bug HERE, combined
//      with zero delays and same-bar Open fills, because it permits a complete
//      round trip inside a bar at prices the rules could only have known after
//      that bar closed.
//   3. The weekly high and weekly close are read from the PREVIOUS completed
//      weekly bar, with an explicit negative shift. The broken version used
//      TimeFrameGetPrice's own defaults - shift 0 and expandFirst - which
//      publish the whole week's aggregate on the week's first bar.
//   4. PositionScore ranks on liquidity, which is known at the decision bar.
//      The broken version ranked on the distance to the week's closing price,
//      which is not known until the week ends.
//   5. Costs exist: commission as a percent of trade value on both legs, and a
//      slippage assumption applied to every fill price.
//   6. The stop uses ExitAtStop = 2, so a gap through the stop is paid at the
//      next bar's opening price rather than at the level you nominated.
//   7. Capital is spread across several positions, and a position is only
//      taken if it is small relative to the symbol's own average turnover.
//   8. The parameters are Param(), not Optimize() defaults copied out of a
//      grid search run on the same data. Choosing them honestly is what
//      Part 31 and Part 32 are for.
//
// ============================ ASSUMPTIONS =============================
//   Universe        must be a POINT-IN-TIME list, including symbols that were
//                   later delisted, merged or renamed. If you cannot build
//                   one, say so in the write-up and treat the result as an
//                   upper bound. Run universe-listing-audit.afl first.
//   Periodicity     Daily.
//   Decision bar    the bar's Close.
//   Fill            the next bar's Open, plus/minus SlippagePct.
//   Commission      CommissionPct of trade value, each leg.
//   Liquidity       50-bar average turnover at the decision bar must clear
//                   MinTurnover, and the intended position must be no larger
//                   than ParticipationPct of it.
//   Stops           max loss, checked on the High-Low range, executed on the
//                   NEXT bar at the regular trade price.
//   Range           run it over as many market conditions as your data allows,
//                   not over the period that first attracted your attention.
// ======================================================================
//
// No result from this formula is a forecast. It is a simulation of a rule over
// one sample of the past, under the assumptions listed above.

AssumedAccount   = Param( "Assumed account size", 100000, 10000, 10000000, 10000 );
PosQty           = Param( "Max open positions", 8, 1, 30, 1 );
BreakoutFrac     = Param( "Breakout fraction of prior weekly high", 1.00, 0.95, 1.05, 0.005 );
MomPeriod        = Param( "Momentum period", 20, 5, 60, 1 );
MomThreshold     = Param( "Momentum threshold %", 5, 0, 25, 1 );
StopPct          = Param( "Max loss stop %", 8, 1, 25, 0.5 );
CommissionPct    = Param( "Commission per trade (%)", 0.10, 0, 1.00, 0.01 );
SlippagePct      = Param( "Slippage per fill (%)", 0.20, 0, 2.00, 0.01 );
MinTurnover      = Param( "Min 50-bar turnover", 1000000, 0, 50000000, 250000 );
ParticipationPct = Param( "Max % of average daily turnover", 1, 0.05, 25, 0.05 );

// ---------------------------------------------------------------------
// 1. Portfolio, costs and execution
// ---------------------------------------------------------------------
SetOption( "InitialEquity", AssumedAccount );
SetOption( "MaxOpenPositions", PosQty );
SetOption( "AllowPositionShrinking", True );
SetOption( "AllowSameBarExit", False );
SetOption( "CommissionMode", 1 );
SetOption( "CommissionAmount", CommissionPct );
SetOption( "UsePrevBarEquityForPosSizing", True );

SetTradeDelays( 1, 1, 1, 1 );

BuyPrice   = Open * ( 1 + SlippagePct / 100 );
SellPrice  = Open * ( 1 - SlippagePct / 100 );
ShortPrice = Open * ( 1 - SlippagePct / 100 );
CoverPrice = Open * ( 1 + SlippagePct / 100 );

PositionSize = -100 / PosQty;

// ---------------------------------------------------------------------
// 2. Higher-timeframe context, read from COMPLETED weekly bars only.
//    A negative shift of -1 means "the previous weekly bar", which had
//    finished before the current daily bar opened.
// ---------------------------------------------------------------------
PriorWeekHigh = TimeFrameGetPrice( "H", inWeekly, -1 );

// ---------------------------------------------------------------------
// 3. Setup, entry and exit. Everything below uses values available at the
//    close of the decision bar.
// ---------------------------------------------------------------------
Turnover    = MA( Close * Volume, 50 );
BaseDollars = AssumedAccount / PosQty;
CapDollars  = ( ParticipationPct / 100 ) * Turnover;

Tradable = Turnover >= MinTurnover
       AND Volume > 0
       AND BaseDollars <= CapDollars;

Momentum = ROC( Close, MomPeriod );

Buy  = Close > BreakoutFrac * PriorWeekHigh
   AND Momentum > MomThreshold
   AND Tradable;

Sell = Close < MA( Close, 10 );

// Rank on something knowable: the more liquid candidate is preferred.
PositionScore = Turnover;

// ---------------------------------------------------------------------
// 4. Risk exit that does not assume a fill at your chosen level.
// ---------------------------------------------------------------------
ApplyStop( stopTypeLoss, stopModePercent, StopPct, 2 );

_SECTION_BEGIN( "Repaired backtest view" );
Plot( Close, "Close", colorDefault, styleCandle );
Plot( PriorWeekHigh, "Previous completed weekly high", colorGreen, styleLine );
PlotShapes( Buy * shapeUpArrow, colorGreen, 0, Low );
PlotShapes( Sell * shapeDownArrow, colorRed, 0, High );
_SECTION_END();

Download impossible-backtest-repaired.afl130 lines

The same trading idea with the fiction removed. Read the “WHAT CHANGED, AND WHY” block against your own list.

=Expected result

Run both on the same universe and range. The repaired version will produce a dramatically smaller result, and may well produce a losing one.

That is the finding. The honest version of a fictional result is usually a much smaller result, and sometimes a negative one. Nothing has been broken by the repairs — the original number simply was not there.

Root causes, ranked

Rank	Fault	Category	Direction
1	Weekly high with default shift	Look-ahead	Enormous, favourable
2	PositionScore from the week’s close	Look-ahead	Large, favourable
3	Decide on close, fill at same bar’s open	Look-ahead	Large, favourable
4	ExitAtStop = 1	Unrealistic fill	Makes the drawdown figure fictional
5	Four parameters optimised on the quoted data	Data snooping	Large, favourable
6	Curated, current-membership universe	Survivorship / selection	Moderate, favourable
7	Zero costs	Unmodelled cost	Moderate, favourable
8	Full position, no participation limit	Sizing	Makes late results impossible
9	Period chosen after the fact	Selection	Unknown, favourable

Notice the pattern in the last column. Every fault biases in the same direction. That is not a coincidence and it is not bad luck: faults that made the result worse would have been found and fixed, because the author would have gone looking. Only the favourable ones survive to be reported.

?Reality check

This is the most transferable lesson in the whole challenge. When you meet a spectacular backtest, the question is never “is there a mistake?” — there is always a mistake somewhere. The question is “why did this particular set of mistakes survive?”, and the answer is almost always that they were the ones that made the number go up.

Apply the same reasoning to your own work. The bugs you find quickly are the ones that hurt your results. The ones that help can sit there for years.

Check your understanding

Question 1. What is wrong with WeeklyHigh = TimeFrameGetPrice( "H", inWeekly ); in a trading system?

• Nothing — it correctly retrieves the weekly high
• The default shift of 0 means the whole week's high is published on the week's first bar, so Monday knows Friday's high
• inWeekly is not a valid interval constant
• It requires TimeFrameSet to be called first

Show the answer and why

Answer: The default shift of 0 means the whole week's high is published on the week's first bar, so Monday knows Friday's high

The documentation states it directly: with shift = 0 compressed data may look into the future, and it gives the weekly-high-on-Monday example. A trading system must use a negative shift to reference completed periods.

Question 2. The report claims a 3% maximum drawdown with a 3% stop and full position size. Which faults make that figure possible? Select all that apply.

• ExitAtStop = 1 fills every stop at the nominated level, so no loss can ever exceed the stop distance
• The look-ahead in the entry condition means losing trades are rare in the first place
• Gaps through the stop are absent from the simulation entirely
• A 3% drawdown is normal for a well-designed system

Show the answer and why

Answer: ExitAtStop = 1 fills every stop at the nominated level, so no loss can ever exceed the stop distance, The look-ahead in the entry condition means losing trades are rare in the first place, Gaps through the stop are absent from the simulation entirely

The drawdown figure is a direct artefact of the fill assumption plus the look-ahead. A drawdown smaller than one stop distance, over 41 full-size trades, is the single most suspicious number in the report — and reasoning from it alone would have got you to the fill assumption before reading any code.

Question 3. The formula optimises four parameters over grids of 11, 36, 20 and 19 values. Roughly how many specifications were evaluated, and against how much data?

• 86 combinations against 41 trades
• Around 150,000 combinations against a sample containing 41 trades
• Four combinations, one per parameter
• It cannot be determined from the formula

Show the answer and why

Answer: Around 150,000 combinations against a sample containing 41 trades

11 x 36 x 20 x 19 is about 150,000. Selecting the best of that many results from a sample of 41 trades guarantees an impressive-looking cell whether or not anything is there. MomPeriod = 17 is the tell: nobody chooses 17 from theory.

Question 4. Why is a curated watchlist worse than an ordinary current-membership index list as a test universe?

• Watchlists cannot be used with the Analysis window
• It carries ordinary survivorship bias plus a selection made by a person who had already seen how those names performed
• Watchlists contain fewer symbols
• Watchlist data is not adjusted for splits

Show the answer and why

Answer: It carries ordinary survivorship bias plus a selection made by a person who had already seen how those names performed

Index membership is at least chosen by a rule. A personal watchlist assembled after the period is a list of names the author noticed — and people notice names that went up. Both problems bias in the same direction.

Question 5. Every fault in this challenge biases the result favourably. Why is that the expected pattern rather than a coincidence?

• Because AmiBroker's defaults are optimistic
• Because faults that made the result worse would have been investigated and fixed — only the favourable ones survive to be reported
• Because backtests are inherently biased upward
• Because small-cap data is unreliable

Show the answer and why

Answer: Because faults that made the result worse would have been investigated and fixed — only the favourable ones survive to be reported

It is a selection effect operating on the researcher rather than on the data. A disappointing result prompts a search for errors; a spectacular one prompts a press release. This is why the question to ask of an impressive backtest is not "is there a mistake?" but "why did these particular mistakes survive?".

Check my answersClear