Context, Evidence and Base Rates
Somebody tells you that a pattern was followed by a higher close ten bars later 57% of the time, across four thousand occurrences. It sounds substantial. It is uninterpretable, and it will stay uninterpretable until you know one more number: how often the close was higher ten bars later on all the other bars. This lesson is about that number, why it is so often missing, and the specific rhetorical move that gets used to avoid computing it.
The base rate
Section titled “The base rate”The base rate is what happens anyway. Concretely, for a pattern study: take every bar of the same instruments, over the same period, that passed the same data-quality filter, and measure the same forward quantity over the same horizon. The distribution you get is the thing your pattern has to beat.
Here is why it matters, using numbers invented purely to show the arithmetic.
Two studies, same conclusion attached, opposite meanings
| Bar | Study A | Study B |
|---|---|---|
Pattern occurrences | 4,000 | 4,000 |
Higher 10 bars later, after pattern | 57% | 57% |
Higher 10 bars later, all bars | 52% | 58% |
Difference | +5 points | −1 point |
The instruments that make up most equity databases spent more time rising than falling over most historical windows, so an unconditional up-rate above 50% is ordinary rather than notable. Reporting 57% without the comparison is not dishonest so much as empty: it does not carry the information that would make it a finding.
Conditional versus unconditional
Section titled “Conditional versus unconditional”Two quantities, and it is worth keeping the names straight:
| Quantity | Means |
|---|---|
| Unconditional | The distribution of forward outcomes over all measurable bars |
| Conditional | The distribution of forward outcomes over bars where the pattern was true |
The effect is the difference. Not the conditional number on its own, not the conditional number compared with zero, and not the conditional number compared with some external benchmark index. The difference between the same measurement made on two sets of bars drawn from the same data.
Four ways this goes wrong in practice, in ascending order of subtlety:
- No benchmark at all. The conditional figure is reported alone.
- A benchmark from a different sample. The pattern is measured on ten years of one instrument, the base rate quoted from a textbook or a different index over a different period. Two numbers that were never comparable.
- A benchmark that does not share the context. The pattern requires an uptrend; the base rate is computed over all bars including downtrends. This one is everywhere, and it is the subject of the next section.
- Overlapping windows counted as independent observations. Four thousand occurrences of a pattern measured over ten-bar forward windows are not four thousand independent facts if many of them are within ten bars of each other.
The context escape hatch
Section titled “The context escape hatch”Here is the conversation, and you will have it with yourself before you have it with anyone else.
You define a pattern precisely, measure it, and find the difference from the base rate is small. The natural response is: of course — the pattern only works in the right context. In an uptrend. On high volume. Near a support level. Not in the first hour of the session. Not during earnings season.
This response is not automatically wrong. Conditional effects are real, and a good mechanism-based hypothesis often specifies a context. What makes it an escape hatch is when it arrives. Added before the measurement, a context is part of the hypothesis. Added after a disappointing measurement, it is a new hypothesis being tested on the same data that just disappointed you — and you will keep adding contexts until one of them works, which, given enough contexts, one will.
Three rules close the hatch:
Rule one: state the context before you look. Write it in the same sentence as the pattern, before running anything. If you cannot say in advance which context you expect the pattern to work in, you do not have a mechanism, you have a hope.
Rule two: count the contexts you tried. Each context variant is another test. Three patterns times four contexts is twelve tests, and the multiple-comparison arithmetic from the previous lesson applies to the product, not to either factor.
Rule three — the important one: benchmark inside the context. If your pattern requires an uptrend, the comparison is not against all bars. It is against all bars that were in an uptrend. Otherwise you cannot tell whether you have discovered something about the pattern or merely rediscovered that uptrends tend to continue for a while, which everyone already suspected.
Three sets of bars, and which comparison answers which question
- All measurable barsThe unconditional base rate. Answers: what happens anyway?
- Bars meeting the context onlyAnswers: what does the context alone buy you?
- Bars meeting context AND pattern shapeAnswers: what did the SHAPE add?
Most published pattern results report the third minus the first. The reality check at the end of this part computes both, side by side, because seeing them differ is the fastest way to understand why the distinction matters.
How large does a difference have to be?
Section titled “How large does a difference have to be?”A difference of two percentage points on ninety observations is not a small effect. It is noise. The arithmetic is simple enough to do in your head once you have seen it.
For a proportion — an up-rate, say — the standard error is the square root of
p × (1 − p) / n. At an up-rate near 50%, which is the pessimistic case, that gives:
| Observations | Standard error | Rough 95% interval half-width |
|---|---|---|
| 25 | 10.0 points | ±19.6 points |
| 50 | 7.1 points | ±13.9 points |
| 100 | 5.0 points | ±9.8 points |
| 250 | 3.2 points | ±6.2 points |
| 500 | 2.2 points | ±4.4 points |
| 1,000 | 1.6 points | ±3.1 points |
| 5,000 | 0.7 points | ±1.4 points |
Read the 100-observation row carefully. With a hundred occurrences of a pattern, an observed up-rate of 57% is compatible with a true up-rate anywhere from about 47% to about 67%. It is compatible with the pattern being slightly unfavourable. A hundred occurrences feels like a lot when you have scrolled through them on a chart, and it is not.
For a difference between two means — average forward return after the pattern versus after the context bars — the same logic applies with the standard error of the difference:
Fragment — not a complete formula
// Standard error of the difference between two independent group means.StdErrDiff = sqrt( VarPattern / CountPattern + VarContext / CountContext );TRatio = ( MeanPattern - MeanContext ) / StdErrDiff;A t ratio below about 2 gives you very little to work with even before the corrections below. Above 2, you still have the multiple-comparison count to apply, and then the overlap problem.
The overlap problem
Section titled “The overlap problem”Suppose your pattern occurs on three consecutive bars and you are measuring ten-bar forward returns. Those three “observations” share nine of their ten bars. They are close to one observation counted three times, and treating them as three independent facts makes your standard error roughly a factor of the square root of three too small — which is to say, it makes a weak result look stronger than it is.
A worked reading, start to finish
Section titled “A worked reading, start to finish”Numbers invented to show the method, not measurements. Suppose a pattern occurred 140 times. The mean 10-bar forward return on those bars was 1.10%, with a standard deviation of 6.8%. The context group — bars meeting the context condition but not the shape — had 2,050 members, a mean of 0.85% and a standard deviation of 6.5%.
The difference is 0.25 percentage points, in favour of the pattern. Now put an error bar on it. The standard error of the difference between two group means is the square root of the sum of each group’s variance divided by its own count:
- pattern term: 6.8 squared, divided by 140, is 0.330
- context term: 6.5 squared, divided by 2,050, is 0.021
- sum 0.351; its square root is 0.59 percentage points
So the difference is 0.25 with a standard error of 0.59, a ratio of about 0.42. It is well inside the noise, and that is before any correction for how many variants were tried or for the overlap between forward windows.
Two follow-up calculations are worth doing every time, because they reframe the whole exercise.
How large would the difference have to be to clear a ratio of 2? About 1.18 percentage points — larger than the entire forward return of the context group. The pattern would have to more than double what the context was already delivering.
How many occurrences would be needed to detect a difference of 0.25 points against that dispersion? Roughly three thousand, not a hundred and forty. That is the number that decides whether a study of this kind is worth running at all, and it is available before you collect a single observation.
The general shape of this result is not specific to the invented numbers. Forward returns over a short horizon have a standard deviation many times their mean, so the sample sizes required to resolve small differences are large. Any pattern study on one instrument over ten years is working with far fewer observations than the arithmetic demands, which is why pattern research is done across universes rather than symbols.
When the mean and the hit rate disagree
Section titled “When the mean and the hit rate disagree”Report both, always, because they fail in different ways.
The mean forward return is sensitive to outliers. A handful of large moves in a group of a hundred can produce a positive average while the majority of occurrences went nowhere or drifted lower. The up-rate — the proportion of occurrences followed by a positive return — is insensitive to size, so it can look unremarkable while a real and useful asymmetry hides inside it.
When the two disagree, the disagreement is the finding. A pattern with a positive mean and a below-benchmark up-rate is describing something that loses small and occasionally wins big; a pattern with a high up-rate and a mean below the benchmark is the reverse. Both are legitimate shapes for a market phenomenon to have, and both are invisible if you report only one number. The median is a useful third figure for the same reason: a mean far from the median tells you the distribution is doing something the mean cannot express.
What tends to happen once a pattern is defined precisely
Section titled “What tends to happen once a pattern is defined precisely”This course will not tell you the answer for your market, your instruments and your period, because it has not measured them and neither has anyone else. What it can tell you is the shape of result to expect, and why.
Three forces all push in the same direction. The eye selects: you noticed the pattern because of the instances that were followed by a memorable move, and the instances that were followed by nothing did not register as instances. The definition tightens: turning “a long lower wick after a decline” into arithmetic excludes many of the bars that felt like examples, and includes many that do not feel like examples at all. And the base rate absorbs a large share of whatever is left, because a pattern that requires an uptrend inherits the uptrend’s own forward tendency.
The consistent experience of people who do this work is that measured effects come out smaller than the visual impression, often much smaller, and frequently indistinguishable from the base rate once the sample size is taken seriously. That is not a claim that no pattern has ever carried information. It is a warning about what the arithmetic usually does to an impression.
Designing a fair pattern test
Section titled “Designing a fair pattern test”Eight decisions. Write all eight down before you run anything, and the write-up is your protection against the version of yourself that will want to move one of them after seeing the result.
The eight decisions, in the order they have to be made
- 1. UniverseWhich instruments, and where did that list come from? A list of today’s index members answers a question about survivors.
- 2. PeriodWhich dates, chosen for a reason other than the result they give.
- 3. Pattern definitionArithmetic, every threshold visible, tie-breaking stated.
- 4. Context definitionStated now, not after the first result.
- 5. Forward measurementWhat is measured, over how many bars, and from which price.
- 6. BenchmarkBoth of them: all measurable bars, and context-only bars.
- 7. Multiple-comparison budgetHow many variants you are allowed before you stop and hold data back.
- 8. What would change your mindThe result that would make you drop the idea, written before you see any result.
Decision eight is the one people leave out, and it is the one that turns the exercise into research. If there is no outcome that would make you abandon the idea, you are not testing it.
Two smaller points that belong to decision five. First, measuring the forward window
requires reading bars that had not printed yet — Ref() with a positive period — which
is legitimate in a study and a serious bug in a rule. Keep the two uses separate in
your head and label them in your code. Second, the last few bars of every series have
no forward window at all and must be excluded, or they will quietly contribute zeros.
When the answer is “no effect”
Section titled “When the answer is “no effect””A test that finds nothing is a successful test. It cost you an afternoon and it saved you from building a system on an idea that would not have supported one. Write it in the log with the same care as a positive result: the definition, the universe, the period, the counts and the numbers. Six months later, when the same idea occurs to you again with a slightly different threshold, the log is what stops you from spending the afternoon twice.
The one thing a null result does not license is the conclusion that the pattern is useless everywhere and always. Your test covered a universe, a period, a definition and a horizon. Another combination might behave differently — which is precisely why the count of combinations you try has to be part of the record.
What changed
Section titled “What changed”You now have a single question to apply to every pattern claim you meet, including your own: compared with what? You know that the comparison must come from the same sample and, where the pattern requires a context, from within that context. You know roughly how many observations it takes before a few percentage points mean anything. And you know the shape of the escape hatch well enough to notice yourself reaching for it.
The next page does the whole thing on real data: one pattern, defined in AFL, measured against both benchmarks, reported honestly.
Check your understanding
Sources for this lesson
4 verified · checked 2026-08-31
- 01AFL Function Reference — Refamibroker.com/guide/afl/ref.html2026-08-31
- 02AFL Function Reference — Cumamibroker.com/guide/afl/cum.html2026-08-31
- 03AFL Function Reference — Nzamibroker.com/guide/afl/nz.html2026-08-31
- 04AFL Function Reference — SafeDivideamibroker.com/guide/afl/safedivide.html2026-08-31
Every technical claim on this page was checked against the official AmiBroker documentation on the date shown. Where the course disagrees with folklore, the source is how you can tell which one to trust.