What a prediction confidence score actually is

Almost every prediction site attaches a confidence figure to its picks, and almost none of them say what the number is. The common explanation — that a 65% confidence means the pick wins 65 times out of 100 similar matches — describes a calibrated probability, which is a demanding thing to produce and a very different object from what most confidence scores actually are.

This article says what the number can be, and what ours is held to, because a number on a page that nobody defines is decoration.

Confidence and probability are not the same thing

A probability is a claim about the world: this outcome happens 65% of the time in situations like this one. It can be checked. Publish enough of them and reality will tell you whether your 65%s came in about 65% of the time — that is calibration, and it is measurable with a proper scoring rule.

A confidence score, as the term is used across this industry, is usually a strength-of-signal indicator: how much the model’s inputs agree, how clean the picture looks, how comfortable the system is. That is a statement about the model, not about the match. It might correlate with being right. It is not a claim that can be checked against a result, because no result can contradict “the model felt sure”.

The distinction matters most when the two get conflated in the reader’s head, which is the default outcome when a site prints a percentage and says nothing further.

How confidence scores are usually built

Across the industry there are broadly three ways to produce the number.

The top probability, relabelled. Take the model’s highest outcome probability and call it confidence. Honest, but redundant — it says nothing the probability did not already say.

The top probability, damped. Take that same figure and reduce it when the model’s inputs disagree with each other — when, say, a team’s recent form points one way and its longer record another. The result sits at or below the probability and carries a little extra information: how united the evidence is behind the pick. The size of the reduction is usually a judgement rather than a measurement.

A separately fitted score. Train something on the model’s own outputs to predict how often picks like this one actually land. This is the only version that can legitimately be read as “the chance the pick is right”, and it needs a large settled sample to fit without fooling itself, because fitting a correction to too few results produces a number that looks rigorous and is noise.

Most sites do not say which of the three they are showing you. Some are showing none of them.

How ours is checked

Ours is called Trust, a score out of 10 on every tip. It is derived from the model’s own probability for the pick, and we do not ask you to take it on faith. Trust is the model’s probability in tenths, so a 74% call is a Trust 7, and the predictions page prints how many Best Tips at each Trust level have won beside the scale. Trust 7 calls should land roughly seven times in ten; if they land five, the page says five.

That is the whole audit, and it is the only sense in which a confidence score means anything: not how it is built, but how it has done.

The consequences of that design, stated plainly

It is not a promise the pick is correct. It is derived from a probability, and a probability is a claim about frequency across many matches, not about any single one.

It cannot be very high, and that is the sport rather than the model. The score inherits its ceiling from the top probability, and in football the top probability is usually modest — a genuinely lopsided fixture might reach the mid-seventies, and a typical one sits near half. If you see confidence figures in the nineties on a football site, the number is not doing the job this one does. The sport does not permit that kind of certainty.

It is only as good as the sample behind it. A Trust level with a few hundred settled picks behind it tells you something; the same level with a dozen tells you nothing yet, and the table shows the count so you can tell the difference.

What to ask of anyone else’s confidence score

Three questions, and they are answerable for a well-built system:

1. Is it a probability or a signal? If the site says a confidence score is the chance of winning, it should be checked against results — ask where those results are published.
2. What is its range in practice? A score that regularly reaches 90%+ on football fixtures is either measuring something other than match outcome, or it is not calibrated.
3. What makes it go down? A confidence score that only ever expresses enthusiasm is a marketing device. A useful one has a defined mechanism for becoming less sure, and its history shows it doing so.

Why we publish the audit at all

Because a confidence number is exactly the sort of thing that is easy to invent and impossible to challenge. Ours can be challenged: check the Trust table on the predictions page, or the calibration page for every probability we have published, misses included, and decide for yourself what the label is worth.

A confidence score is a hint about how much the model agrees with itself. It is not a promise, and no number on any prediction site is. Stake only what you can afford to lose.