Hot Streaks and Survivorship

Take 500 players who are all exactly average, with no hot or cold ability whatsoever. Ask how many had a stretch of ten straight games above their own average. The answer is not zero — it is a substantial number, because with 500 players and a full season of possible starting points, you have given yourself tens of thousands of chances to find one.

The streak you found is genuine. What is not genuine is the implication that the player has a property called "hot," because you would have found a streak like that whether or not anyone did.

Related trap, different mechanism. When a claim only counts the cases that made it to the end, the ones that dropped out are invisible.

• "Players who start hot finish strong" — players who start cold get benched, so they never generate a second half to count.
• "This system has hit 12 straight" — the systems that missed were quietly discarded, and you only ever see the one that survived.

The honest version always reports the denominator: how many candidates were examined, not just how the winner did.

Ask: *how many things were looked at before this one was chosen?*

If the answer is one, and it was chosen in advance, the finding is worth something. If the answer is "we scanned the league for the most striking pattern," then the pattern being striking is the reason it was selected, not evidence that it means anything.

This is the same discipline we hold our own research to. When we tested whether MLB prop prices reprice at a particular hour, we ran the same test on starting pitchers — whose lineups are known days ahead — specifically so a result that showed up in both would be thrown out rather than published.