Model

Can ChatGPT predict football matches?

Published

Can ChatGPT predict football matches? Ask it and you will get an answer — a scoreline, a winner, often a percentage and a paragraph of reasoning that reads like analysis. The answer arrives in seconds, for any fixture in any league, and it never says it does not know. That last property is the problem, and it is the one worth understanding before you treat a language model’s output as a prediction rather than as text about a prediction.

This is not an argument that language models are useless for football. It is an argument about what kind of object their output is.

What a language model is doing when you ask it for a score

A large language model generates the next token that best fits the text so far, given everything it learned from its training data. When the text so far is “Predict the result of Team A vs Team B,” the best-fitting continuation is a passage that looks like football punditry — because millions of such passages exist in its training data. The output is shaped by what football analysis sounds like, not by a probability distribution over match outcomes.

Three consequences follow directly, and none of them are fixable by better prompting.

There is no distribution underneath. A statistical match model computes an expected goal rate for each side, then integrates over every possible scoreline to get home/draw/away probabilities that sum to one and are internally consistent with its over/under and both-teams-to-score numbers. A language model writes “I’d say around 55% for the home side” because that phrasing fits. Ask it for 1X2 percentages and the three numbers may not sum to 100. Ask it separately for the same match’s over 2.5 probability and there is no guarantee the two answers are compatible — they were generated independently, not read off one model.

The knowledge has a cut-off, and the fixture does not. The model’s parameters are frozen at training time. Squads, form, injuries and even league membership drift away from that snapshot continuously. Tools that let a model browse or call an API patch the input, but the patch is a paragraph of retrieved text, not a re-estimated set of team ratings.

The answer is not stable. Run the same request twice and you can get a different scoreline, a different winner, a different confidence. A number that changes when nothing about the match changed is not an estimate of anything.

The number that sounds like a probability

The most misleading thing a language model produces is the percentage. “62% chance” reads exactly like the output of a calibrated model, and it is doing something entirely different: it is a rhetorical marker of confidence, sampled from how confident text usually sounds in that context.

The distinction is testable, and the test is calibration. Take every occasion a source said 60%, and check what fraction of those events actually happened. A calibrated model lands near 60%. Text that merely sounds confident has no mechanism forcing that relationship to hold — and a source that says 70% for both a tight derby and a runaway favourite is not measuring anything; it is expressing a mood.

You cannot fix this by asking the model to be honest about its uncertainty. Sincerity is not the missing ingredient. The missing ingredient is a scoring rule applied to a frozen record.

The test that settles it: can the answer be scored?

The Brier score is the standard way to grade probabilistic predictions. For a single outcome it is the squared difference between the probability you gave and what happened, where the outcome is 1 if it occurred and 0 if it did not.

Suppose a source says a home win is 70% likely. If the home side wins, the Brier contribution is (0.70 − 1)² = 0.09. If they do not, it is (0.70 − 0)² = 0.49. Average that across hundreds of predictions and you have a single number that punishes both overconfidence and vagueness: always saying 50% earns a flat 0.25, so any source that cannot beat 0.25 is adding nothing to a coin flip.

Now try to run that calculation on a chat transcript. To score a set of predictions you need four things:

  1. A prediction that was recorded before kick-off, not reconstructed afterwards.
  2. A probability, not a scoreline — “2–1” cannot be scored against the outcome space of a match.
  3. A complete set, including the fixtures where the answer looked bad. A record assembled from screenshots people chose to keep is a record of the ones that worked.
  4. A stable source identity. If the underlying model is updated between predictions, the record grades a moving target unless each prediction carries the version that produced it.

An ad-hoc chat conversation supplies none of these by default. This is why “AI football predictions” as a category is so hard to evaluate: the product is a paragraph, and paragraphs do not have a hit rate. The claim can never be wrong because it was never precise enough to be wrong.

What the alternative actually requires

The reason a dedicated prediction model can be graded is not that it is smarter. It is that it is boring in the right places. Every prediction is generated from a fixed set of inputs, written down before the match with a model version attached, and never edited afterwards — so the record can be totalled at any time by anyone, including on the fixtures where the model was badly wrong. That is what our public track record is, and the methodology page sets out what the model uses and where it fails.

None of that makes a model good. A frozen, complete, scored record makes it checkable — a strictly lower bar, and the only one that can be verified from outside. You do not have to take anyone’s word for it.

Where a language model genuinely helps

Dismissing the tool because it is bad at one job would be its own kind of sloppiness. Language models are genuinely useful around football prediction, just not as the predictor:

  • Explaining a market you do not understand. Asian handicap quarter-lines, how a push works, what an overround is — this is textbook material and the model has read all of it.
  • Writing the code for your own model. Fitting a Poisson or Dixon-Coles model, pulling a results table into a dataframe, computing a Brier score across a season — a language model will write that in minutes.
  • Turning numbers you already trust into prose. Given a real probability set, it is good at describing it. The failure mode only appears when it is asked to invent the numbers.
  • Interrogating a claim. Paste a tipster’s “87% accurate” banner and ask what would have to be true for that to be meaningful. It is a better sceptic than it is a forecaster.

The pattern: use it where the input is knowledge that already exists in text, not where the input is an estimate that has to be computed.

The practical checklist

If you see any AI-branded football prediction, including one produced by a wrapper around a chat model, three questions separate a system from a paragraph generator:

Is the same output produced twice? If asking again changes the answer, there is no model underneath, only sampling.

Do the probabilities cohere? Home, draw and away should sum to 100%. The over/under and both-teams-to-score numbers should be consistent with the same scoreline distribution. Incoherence is the clearest tell that the numbers were written rather than computed.

Is there a record, and does it include the losses? Not a testimonial, not a highlight reel — a complete, dated, pre-committed list with a scoring rule applied to it. If the answer is no, the accuracy question is not merely unanswered; it is unanswerable.

So: can ChatGPT predict football matches? It can produce a fluent, plausible, instantly available answer about any fixture on earth. What it cannot do is produce an answer that can be checked — and in a field where the honest edge over the market is small and hard-won, checkability is the entire distinction between a forecast and a sentence.

Whatever you read, from a model or a person, football remains uncertain by construction. Stake only what you can afford to lose.