All lessons

Reading the Numbers

Which Stats Stabilize First

5 min read

TL;DR

Not all statistics become trustworthy at the same rate. Some are meaningful within weeks; others need more than a season. Knowing which is which tells you what you are allowed to conclude in May.

Stabilization is about how much is skill

A statistic stabilizes quickly when most of its variation comes from the player and slowly when most of it comes from circumstance.

Strikeout rate for a pitcher is largely about the pitcher, so it becomes informative fast. Batting average on balls in play depends heavily on where fielders happen to be standing, so it takes an enormous sample before it says anything about the hitter.

The rough ordering

Within a sport, the ordering is more stable than any exact threshold:

  • Fastest: rates that reflect a repeatable action — strikeout rate, walk rate, three-point attempt rate, snap share.
  • Middle: contact and quality-of-contact measures, shooting percentages.
  • Slowest: outcome rates that depend on other people — batting average, batting average on balls in play, team-dependent counting stats, anything involving touchdowns.

The general rule: the more people involved in producing the number, the longer it takes to mean anything about any one of them.

What to do with a slow stat early

Do not discard it. Regress it, hard.

A slow-stabilizing statistic in a small sample should be shaded strongly toward the player's own prior — his career rate, or the population average for his role. As the sample grows, the weight on the observed number rises and the weight on the prior falls.

That is precisely the operation our NFL Draft Kit baseline performs: a player with four games gets pulled hard toward the positional mean, and a player with seventeen barely moves.

Keep these

  • Stats stabilize fast when the player controls them and slowly when others do
  • Strikeout and walk rates stabilize long before batting average
  • A slow stat in a small sample should be regressed, not discarded

Up next

Expected Stats vs What Happened