Backtest forensics · Aug 2026
An X thread said the strongest 52-week-high breakouts are the ones to buy. It ran on today’s index members applied backward — so I re-ran the whole thing on point-in-time data, with every delisted name put back where it belongs.
The setup, from @spluscollective: every time a stock in the S&P 500 or Nasdaq 100 closes at a new 52-week high, measure its return over the next three months. Segment the breakouts by how far the closing price punched above its 21-day EMA, in units of ATR. The finding was clean: the stronger the breakout, the better it pays and the less often it pulls back — so stop waiting for the dip and just buy.
It’s a good study. But it has one load-bearing flaw, stated right in the first line: “all 511 stocks in $SPY and $QQQ.” That’s today’s membership, applied across 14 years of history. Every company that was kicked out of the index — the blowups, the busts, the ones acquired on the way down — is silently deleted from the sample. That’s survivorship bias, and it doesn’t hit evenly. It hits the violent-breakout tail hardest, because a former high-flyer that ripped +4 ATR and then collapsed out of the index is exactly the data point that gets erased.
So I rebuilt it the honest way: the full “Current & Past” universe including delisted tickers, and a breakout only counts if the stock was an actual index member on that date (Norgate’s point-in-time constituency). Same rule, same three-month hold, same ATR buckets. Here’s what changes.
All breakouts · their method
avg 3-mo return, today’s members. Their thread reported +3.68% — a near-exact reproduction.
All breakouts · survivorship-safe
Put the delisted names back and the average edge falls by ~1 point.
The “best breakout” bucket (>3.5 ATR)
+6.05% → +3.89%. A third of the flagship edge was bias.
My biased re-run lands almost exactly on the thread’s numbers — same monotonic ladder, same falling pullback rate — which tells me the reproduction is faithful. Then swapping in point-in-time membership flattens the ladder. The two weakest buckets barely move. The two strongest give back most of their shine, and the flagship “>3.5 ATR” bucket loses 2.17 points, over a third of its claimed edge.
their method (today’s members) vs. survivorship-safe (point-in-time)
| ATR bucket | n | % pulled back | avg 3-mo | median | win % | % ≥20% |
|---|---|---|---|---|---|---|
| < 1.5 | 570 | 88% | +2.76% | +2.80% | 61% | 6% |
| 1.5 – 2.5 | 6,767 | 84% | +2.48% | +2.54% | 61% | 5% |
| 2.5 – 3.5 | 3,717 | 75% | +2.67% | +2.63% | 60% | 7% |
| > 3.5 | 1,191 | 60% | +3.89% | +3.44% | 62% | 10% |
| All | 12,245 | 80% | +2.69% | +2.66% | 61% | 6% |
Here’s the part of the thread that does survive de-biasing, and it’s the practically important one. Strong breakouts rarely come back: in the honest data, only 60% of >3.5-ATR breakouts ever tag the 21-EMA within a month, versus 88% of the timid ones. If your plan is to wait for the pullback, you systematically miss the best names — because the best names don’t pull back. The “just buy the breakout” conclusion holds. It’s the size of the reward that was oversold, not the direction of the advice.
within one month · survivorship-safe · lower = rarely gives you the dip entry
Splitting the honest sample by index is the most interesting thing the original thread couldn’t see. The breakout edge is concentrated in the Nasdaq 100: its members average +4.09% over three months versus +2.63% for the S&P 500, and the strong-breakout bucket pays +6.72% in the Nasdaq versus +3.58% in the S&P. Higher-beta growth names carry the effect. In the S&P alone the ATR ladder is nearly flat — the “stronger breakout” premium is a large-cap-growth phenomenon wearing a market-wide costume.
avg 3-month return · point-in-time members of each index
An event study isn’t a strategy, so I wired the honest rule into an actual portfolio: buy every fresh breakout (point-in-time member, no new high in the prior 21 days), hold 63 trading days, rank by ATR distance so that when signals exceed capacity the strongest win the slot, 10 bps/side of friction. It’s a legitimate long-only breakout book — and it exposes the catch the thread’s per-trade framing hides.
| Variant | CAGR | MaxDD | MAR | Sharpe | Profit factor | Avg exposure |
|---|---|---|---|---|---|---|
| Buy all breakouts, hold 3mo | 12.1% | −31.1% | 0.39 | 0.75 | 1.78 | 94% |
| + S&P>200-day regime gate | 11.5% | −26.2% | 0.44 | 0.76 | 1.81 | 90% |
| Only >3.5-ATR breakouts | 6.6% | −20.4% | 0.32 | 0.67 | 2.00 | 38% |
The “>3.5 ATR” bucket has the best per-trade edge and the worst portfolio. There simply aren’t enough violent breakouts to stay invested — gate on them and you sit in cash 62% of the time.
ATR-breakout size is a superb ranker, a lousy filter.So the correct way to use the thread’s headline finding isn’t to only buy the monster breakouts — it’s to take them all and let the ATR score decide who gets the last slot when breakouts are plentiful. A market-regime gate (only open new positions while the S&P is above its 200-day average) is the bigger lever for a real book, trimming the 2022 drawdown from −31% to −26% for almost no give-up in return. The honest ceiling here is a MAR around 0.44 — a respectable momentum sleeve, not the effortless money the per-trade averages implied.
C > HHV(H,252)[1]), de-duplicated to fresh events (no new high in the prior 21 bars). ~18,000 raw events; 12,245 pass point-in-time membership.(Close − EMA21) / ATR(14), Wilder ATR. Buckets <1.5 / 1.5–2.5 / 2.5–3.5 / >3.5, matching the thread.The takeaway isn’t “the thread was wrong.” It nearly reproduced, and its core advice — buy the breakout, don’t wait for a dip that never comes — is right. The takeaway is that the number you get when you delete the losers is not the number you’ll trade. Point-in-time data is the difference between a finding and a fantasy.