The Brier score: how to judge a football prediction model
Published
Here’s an uncomfortable truth about football prediction: a model that just picks the home team every time gets about 44% of matches “right” (43.6% in our eight-league dataset). So when a tipster brags about 50-something percent accuracy, the honest question isn’t “is that high?” — it’s “compared to what, and how confident were the calls?”
The tool that answers both questions is the Brier score, and it’s the number we hold our own model to on the public track record.
What it measures
A prediction isn’t a pick, it’s a set of probabilities: say 55% home, 25% draw, 20% away. The Brier score compares those probabilities with what actually happened, treated as 100%/0%/0%. For each outcome you take the difference, square it, and sum:
- Home win happens: (0.55 − 1)² + (0.25 − 0)² + (0.20 − 0)² = 0.305
- Away win happens instead: (0.55 − 0)² + (0.25 − 0)² + (0.20 − 1)² = 1.005
Lower is better. Confident and right scores near 0; confident and wrong gets punished hard; timid predictions land in the middle. Averaged over hundreds of matches, luck washes out and calibration remains.
What good looks like in football
Numbers only mean something against benchmarks. For 1X2 in top European leagues:
| Predictor | Typical Brier (3-outcome) |
|---|---|
| Always 33/33/33 | ~0.667 |
| Historical league frequencies (≈44/25/31) | ~0.65 |
| A competent statistical model | 0.57–0.60 |
| De-vigged closing betting odds | ~0.55–0.56 |
Two lessons hide in that table. First, the gap between “naive” and “excellent” is small in absolute terms — football is genuinely hard to predict, and anyone claiming huge margins is selling something. Second, the closing market sits at the bottom — it’s the strongest public predictor in existence, which is why we don’t sell “value bets” against it.
Our own numbers, since we’re asking
In our pre-launch backtest over 2,071 matches (three seasons, walk-forward, no peeking), our model scored 0.578 against 0.652 for the frequency baseline and 0.555 for the closing odds. Better than naive, behind the market — exactly where a transparent statistical model should honestly expect to sit, and the live track record shows whether the gap closes as the model improves. The full table is on the methodology page.
Why we show it next to every accuracy figure
Accuracy answers “how often was the top pick right?” — a fine headline, but easy to game with safe favourites. Brier answers “were the probabilities worth anything?” A site quoting only accuracy is showing you the flattering angle. When you evaluate any prediction service — ours included — demand both numbers and the sample size. On our track record all three are on every card, updated automatically after every matchday.