Method — how the CBB model works

How it works

A win-probability model for college basketball — and a backtest built to be honest.

This model predicts the winner of NCAA Division I men's basketball games. The harder problem isn't the prediction — it's proving the prediction would have worked. Sports models are notoriously easy to fool: feed them a little information from the future and last season's results look uncannily predictable.

So the whole pipeline is built around one rule — every game is forecast using only what was known before tip-off— and the numbers on the overview page are the result: out-of-sample accuracy, calibration, and a head-to-head against the betting market across two complete seasons. This page explains how it's built and why the backtest can be trusted.

The pipeline

Four stages, run forward through time.

1FeaturesEach game described by 44 features — ratings, form, roster strength, travel — all computed before tip-off.

→

2Walk-forwardThe model is retrained every calendar day on prior seasons plus games already played — never the future.

→

3PredictXGBoost returns a win probability, averaged over both orientations to cancel home/away bias.

→

4ScoreEvery prediction is graded against the result: accuracy, Brier, calibration and a market head-to-head.

Each game is predicted as if live, then graded once it's played · read left to right

Step 1 · Features

Every feature is frozen as it stood before tip-off.

Each game is turned into 44 features describing the two teams: prior-season efficiency and ranking, running form within the current season (win rate, scoring margin, last-ten record, strength of schedule), eligibility-adjusted roster strength, home court, and travel distance and time-zone shift.

The critical detail is wheneach number is measured. Season-form features are computed by stepping through games in chronological order and locking each game's features beforeupdating the running totals — so a November game only ever sees October's games, never March's. Roster strength blends prior-season player ratings with a Bayesian update as the season unfolds. Nothing from the game being predicted, or any later game, can touch its features.

This is the guard against data leakage — the most common way a sports backtest quietly cheats. Get this wrong and the accuracy on this site would be fiction.

Step 2 · The model

An XGBoost classifier, predicted from both sides and averaged.

The features feed an XGBoost gradient-boosted tree model that outputs a single number: the probability the home team wins. Trees handle the non-linear interactions between ratings, form and context without hand-built rules.

One subtlety: a model can develop a small bias toward whichever side is labelled "team A." To cancel it, each game is predicted twice — once as listed and once with the teams swapped — and the two are averaged. This symmetric prediction removes positional bias and measurably improved the scores.

p(home) = ½ · [ p(home) + (1 − p_swap) ]p_swap = the model run with the two teams exchanged

Step 3 · Walk-forward backtest

The model is retrained every day, on the past only.

Rather than train once and test on a held-out chunk, the backtest walks forward through the calendar. To predict a given day's games, the model is retrained on all prior seasons plus every game already played that season — and nothing after. The next day, that day's results join the training set and the model retrains again.

This mirrors exactly how the model would have been used in real time, and it's why each of the ~11,000 predictions is genuinely out-of-sample. The trade-off is cost — hundreds of model fits instead of one — but it's the only way to get an accuracy number you can believe.

Validation

Out-of-sample, the model clears the naive baselines and runs close to the market.

The honest scoreboard: the model beats "always pick the home team" and a coin flip comfortably, and lands just behind the betting market's closing line on the games where odds exist. The market is a famously efficient baseline — running step-for-step with it out-of-sample is the real result here.

Market line71.8%

This model71.6%

Always pick home64.0%

Coin flip50.0%

≋ Scale starts at 45%, not 0 · longer is better. Accuracy is the share of games where the higher-probability side won. The market and the model are measured on the same subset of games that carry odds.

Win-prediction accuracy · 2025 + 2026 seasons (~11,000 games) · point-in-time, walk-forward · market = vig-free closing line on the subset with odds

How it's built

A reproducible pipeline — and this dashboard only ever reads its published output.

IngestESPN + BarttorvikGames, scores, team efficiency and player data into a SQLite warehouse.

→

ModelPython + XGBoostPoint-in-time features, walk-forward retraining, symmetric prediction.

→

PublishJSON blobsThe backtest aggregated into six small files — no raw data leaves the model repo.

→

ServeThis dashboardA Next.js front end renders the published backtest.

PythonXGBoostSQLiteNext.jsTypeScriptVercel

The whole system is hand-built: the data warehouse, the feature engineering, the walk-forward backtest harness, the publish step and this dashboard — the same data-modelling, pipeline and visualisation work that sits behind any production analytics system.

The model lives in a separate repo; the dashboard renders the aggregated backtest it publishes

Limits

What it can't do.

The market is a hard baseline.On the games with odds, the closing line edges the model on raw accuracy. The model's value is in specific spots where it disagrees with the book — not in beating the line everywhere.

Probabilities, not prophecies.A 70% favourite still loses about three times in ten. A wrong call on a confident game is usually variance, not a broken model.

Two seasons is a finite sample.Roughly 11,000 games is a solid test, but two seasons of college basketball still carry real year-to-year variation. Treat small edges as noise.

Some things aren't modelled.Injuries beyond what roster availability captures, coaching changes, in-game momentum and motivation never enter the math. This is a calibrated baseline, not the final word.

Key terms

A quick statistical glossary.

XGBoostA gradient-boosted decision-tree model: it builds hundreds of small trees in sequence, each one correcting the errors of the last. It is the workhorse for tabular prediction problems like this one and handles non-linear interactions between features automatically.

point-in-timeEvery input is frozen as it stood before the game tipped off. Season form, ratings and roster strength are computed from games already played — never from the game being predicted or anything after it. This is what makes a backtest honest rather than a memory test.

data leakageWhen information that wouldn't have been available at prediction time sneaks into the features — e.g. using end-of-season ratings to 'predict' a November game. Leakage inflates backtest accuracy and is the single most common way sports models fool their builders. This pipeline is built specifically to avoid it.

walk-forward validationInstead of one train/test split, the model is retrained as time moves forward: to predict a given day, it trains only on prior seasons and the games already played that season. It mirrors exactly how the model would have been used live.

Brier scoreThe mean squared error of a probability forecast: average of (probability − outcome)². Lower is better; 0 is perfect, 0.25 is a coin flip. It rewards being both right and appropriately confident.

AUCArea under the ROC curve: the chance that a randomly chosen winner was given a higher probability than a randomly chosen loser. 0.5 is no skill, 1.0 is perfect ranking. It measures ordering, independent of calibration.

calibration (ECE)Whether stated probabilities match reality: of the games called 70%, do about 70% actually happen? Expected Calibration Error is the average gap between predicted and observed rates across probability bins — smaller is better.

vigThe bookmaker's built-in margin: turn a book's odds into probabilities and they sum to more than 100%. To compare the model and the market fairly, that excess is stripped out and the two sides rescaled to a clean 100% — the 'vig-free' line used throughout.

Built by Isaac Kaczor. The model and this dashboard are independent hobby work — not affiliated with the NCAA or any betting operator. Forecasts are for interest and discussion only.