Goodness-of-Fit: R², AIC, and BIC

Goodness-of-fit measures how well a model explains the variability in the data. Three you'll see most often:

R² (Coefficient of Determination)

The proportion of variance in the outcome explained by the predictors. Ranges from 0 to 1.

• R² = 1: the model explains all variance (every prediction is perfect).
• R² = 0: the model explains nothing (you're no better than predicting the mean).

Critical limitation: Add more predictors and R² almost never decreases. It always "rewards" complexity, even if the extra variables are noise. R² alone will favor the most complex model, which can be problematic, as you lose understandability when your models are more complex.

AIC and BIC

Both metrics balance goodness-of-fit against model complexity by penalizing models with more parameters. Lower values = better model.

• AIC (Akaike Information Criterion): Lighter complexity penalty. Tends to favor slightly more complex models. Use when prediction is the goal.
• BIC (Bayesian Information Criterion): Heavier complexity penalty, especially as sample size grows. Favors simpler models more aggressively. Use when explanation and parsimony are the goal.

In practice, you may choose to report R² alongside AIC or BIC. R² tells you how well the model fits. AIC/BIC tell you whether the fit is worth the complexity.