Can We Beat the Over/Under?

We found something real: bad weather knocks points off a game's final score, even though it doesn't change who wins (that's the first post if you missed it). The obvious next step is to try to actually predict it — so we built a model, tested it honestly, and now run it against this week's real games. Here's how it works, including the parts that didn't pan out.

What the model actually looks at

Strip away the machine-learning language and it's a pretty simple idea: to guess how many total points two teams will score, you mostly need to know how good their offenses and defenses are, and whether the weather is going to get in the way.

Under the hood it's a "random forest" — picture a few hundred simple decision trees ("if this team's offense rating is above X and it's raining more than Y inches, guess roughly Z points"), each one a little different, averaged together. No single tree is very smart on its own; the average of a few hundred slightly different ones tends to be pretty solid.

We tried making it smarter. It didn't work.

Before settling on the simple version, we tried feeding it a lot more: how run-heavy or pass-heavy each team's play-calling is, a deeper measure of defensive disruptiveness, how many days of rest each team had. More inputs, more nuance, should mean a smarter model — right?

Turns out no — it came back slightly worse, not better. The extra stats we added (play-calling balance, defensive disruptiveness) are already closely related to the team-strength rating we started with, since they're measuring overlapping things. Piling on more overlapping numbers just gave the model more ways to chase noise in the historical data instead of learning something new. So the version running on the site today is the plain one: team strength plus weather, nothing else. Simple won.

How we tested it honestly

The trap with any prediction model is testing it on the same games it learned from — that's like grading a student on the exact practice test they studied from and being impressed they got a good score. Instead, we trained it only on older seasons, then had it guess games from more recent seasons it had never seen. That's the fair version of the test, because it's the same situation the model is actually in every week: guessing a game that hasn't happened yet.

On that fair test, the model's guesses are typically within about 12 points of the real final total. For context, a typical major-conference game scores somewhere in the 45–55 point range combined — so this isn't a precision instrument, but it's clearly picking up on real signal rather than guessing randomly.

So does it actually beat the sportsbooks?

This is the more interesting and less settled question, and we're not going to oversell it. Sportsbooks set their own total for every game, and they're generally quite good at it — that's their whole business. We compared our model's guesses to the actual posted lines for real games since 2017 (the era with solid sportsbook data) and found something suggestive but not proven: when our model and the market disagree, the real outcome tends to lean slightly toward our model's direction in bad weather — but not by enough to call it a confirmed edge yet, statistically speaking. More seasons of data will make that answer clearer over time.

See it guess this week's games

Every week, we run this model against the real upcoming schedule and list its guess right next to the actual posted over/under, on the dashboard. "Edge" is just the gap between the two — how much the model and the market disagree, in points. A big edge is interesting to look at. It is not a betting recommendation.

What we'd want before trusting this more

A model that's merely "not obviously wrong" isn't the same as one you should trust. Some real limits worth naming: SP+ ratings early in a season are based on very little data and can be shaky; weather forecasts get less reliable the further out they are; and the "does it beat the market" question needs more accumulated seasons before it's a real answer instead of a hopeful one. We'll keep tracking all of it here as it comes in.