OLS and the Inferential Toolkit

Chapter 11 | Section 11.1

Docker image: ml4t

This notebook shows what classical inference looks like before we leave it behind. Using the same ETF features and labels as the rest of Chapter 11, we fit a statsmodels OLS model and walk through the full inferential toolkit: coefficient significance, Gauss-Markov diagnostics, and robust standard errors.

Learning objectives

• Interpret a statsmodels OLS summary: coefficients, standard errors, t-statistics, p-values
• Test the spherical-errors condition on a panel: heteroscedasticity and residual autocorrelation
• Identify multicollinearity via Variance Inflation Factors (VIF)
• Separate what robust standard errors repair from what they cannot
• Understand why inference diagnostics do not answer the prediction question

Book reference

Section 11.1 ("From Inference to Prediction"). The next notebook, 02_regularization_paths, transitions to the prediction-oriented framework.

Prerequisites

• Familiarity with linear regression and the Gauss-Markov assumptions.
• Conceptual exposure to heteroscedasticity, autocorrelation, and VIF.
• ETF case-study features at case_studies/etfs/features/financial.parquet (Ch8 output) and labels at case_studies/etfs/labels/fwd_ret_21d.parquet (Ch7 output).

Setup

Load Features and Labels

We use the same ETF case study data as the rest of Chapter 11: financial features from Ch8 and 21-day forward return labels from Ch7.

Ch8 builds the skip_recent and mom_accel features as differences of return horizons, so skip_recent_12_1 is ret_252d - ret_21d. Keeping both the raw returns and the derived differences makes the design matrix singular, which is a violation of the no-perfect-multicollinearity assumption severe enough that the robust covariance estimators below cannot be computed at all. We drop the five composites.

Select a Single Training Fold

Inference is an in-sample exercise — we fit one model on one training window and examine its properties. We use the first walk-forward fold and hold back the validation set for a prediction comparison at the end.

Section 1: Statsmodels OLS Summary

Statsmodels provides the full inferential output that scikit-learn omits: coefficient standard errors, t-statistics, p-values, and model-level diagnostics.

Reading the summary: The coefficients table shows many features with small p-values in the P>|t| column, which might seem to indicate "significant" predictors. But the overall $R^2$ is very low — typical for cross-sectional return prediction. A large sample (hundreds of thousands of observations) makes even tiny effects "significant" in the statistical sense, while the economic magnitude may be negligible. Note that even after removing the five exact linear combinations, the remaining features are still pervasively correlated (as VIF will confirm), so coefficient estimates are unstable across samples.

Crucially, these standard errors assume spherical errors: constant variance, uncorrelated across observations. We now test that condition, and the no-perfect-multicollinearity condition alongside it.

Section 2: Gauss-Markov Diagnostic Battery

Section 11.1 states the four Gauss-Markov assumptions: linearity in the parameters, strict exogeneity, no perfect multicollinearity, and spherical errors. When all four hold, OLS is the Best Linear Unbiased Estimator (BLUE).

Only two of the four are testable from residuals. Spherical errors decompose into constant variance (Breusch-Pagan, below) and no correlation across observations (residual autocorrelation, below); multicollinearity is read off the design matrix via VIF. Linearity and exogeneity are assumptions about the data-generating process that residuals cannot confirm — a misspecified model can produce well-behaved residuals.

We also test normality, which is not a Gauss-Markov assumption. It is required for exact finite-sample t and F distributions, not for BLUE.

Heteroscedasticity: Breusch-Pagan Test

Tests whether the error variance depends on the regressors. Financial returns exhibit time-varying volatility (volatility clustering), so we expect rejection.

Correlated Errors: Residual Autocorrelation Within Each Asset

This is a panel: many symbols observed on the same dates. The residual vector is stored in date order, so its neighbouring entries are different assets on the same day, not consecutive observations of one asset. A Durbin-Watson or Breusch-Godfrey statistic computed on that vector measures cross-sectional dependence and reports it under the name of serial correlation.

To test serial correlation we regroup the residuals into one time-ordered series per symbol and measure autocorrelation within each. Two lags are informative here. Lag 1 asks whether yesterday's error predicts today's. Lag 21 sits at the label horizon: fwd_ret_21d is a 21-day forward return, so consecutive daily observations of one asset share 20 of 21 days of outcome window. That overlap induces autocorrelation by construction out to lag 20 and is the dominant source of the correlation we find.

Normality: Jarque-Bera Test

Normality of the errors is not one of the four Gauss-Markov assumptions, and OLS remains BLUE without it. It is what makes the t and F statistics follow exactly those distributions in a finite sample. At this sample size the central limit theorem delivers approximate normality of the coefficient estimates regardless, so a rejection here is the least consequential of the diagnostics in this section.

Financial returns have fat tails and excess kurtosis, so this will almost certainly reject.

Multicollinearity: Variance Inflation Factors

VIF measures how much each coefficient's variance is inflated by correlation with other features. VIF > 5 is a common concern threshold; VIF > 10 indicates severe multicollinearity. Financial features (momentum at different horizons, volatility measures) are pervasively correlated.

High VIF confirms what Section 11.1 discusses: financial features are pervasively correlated. Momentum at 5-day and 21-day horizons, different volatility measures, and volume-price indicators share overlapping information. OLS distributes weight arbitrarily among correlated features, producing unstable coefficient estimates that flip sign across samples.

Section 3: When Standard Errors Fail — Robust Alternatives

Robust standard errors address exactly one failure: non-spherical errors. The point estimates are unchanged and, if linearity and exogeneity hold, they stay consistent; only the variance estimate around them is repaired.

They do nothing for the other two assumptions. If the model omits a relevant variable or gets the functional form wrong, exogeneity fails, the coefficients themselves are not consistent, and a robust standard error is a more careful statement about the wrong quantity. Section 11.1 makes this point directly: when the model is misspecified, the unbiasedness guarantee fails and the coefficients stop being interpretable as structural parameters. No covariance estimator recovers it.

Comparing Standard Error Estimates

Each estimator below assumes some pairs of residuals are uncorrelated, and the question is whether the diagnostics above have already shown that assumption to be false.

• OLS (default): every pair uncorrelated, and equal variance throughout
• HC3: still every pair uncorrelated, but variance may vary with the regressors
• Clustered by date: any two observations sharing a date may correlate; different dates may not
• Clustered by date and symbol: adds any two observations of one symbol; still assumes independence when the symbol and the date both differ
• Driscoll-Kraay: no independence assumption within a window of nearby dates, across the whole cross-section

That last assumption is the one this panel breaks in a way the others do not. fwd_ret_21d is a 21-day forward return, so one symbol on date t and a different symbol on date t+5 share sixteen days of outcome window, and whatever moved the market in those sixteen days is in both residuals. Two-way clustering treats that pair as independent. Driscoll-Kraay does not: it sums the moment conditions across the cross-section within each date and then applies a Newey-West correction over dates, so arbitrary cross-sectional correlation and serial correlation out to the lag length are both admitted. We set that lag to the label horizon, which is where the mechanical overlap ends.

Plain HAC (Newey-West) is the usual answer to autocorrelation, but it reads its lags off the row order of a single time series. On a pooled panel stored in date order those lags run across the cross-section, so it does not estimate what its name promises here. It becomes the right tool once the panel is collapsed to one observation per date, which is what the IC calculation below does.

The features whose robust standard errors differ most from their OLS ones. Reading left to right across the SE columns shows what each dependence the estimator admits costs in precision.

Coefficients with Panel-Robust Confidence Intervals

Rank features by absolute coefficient magnitude and show the Driscoll-Kraay confidence interval around each estimate. The interval width is set by the CONF_Z multiplier below; a bar that crosses the dashed line at zero is a coefficient whose sign the data does not pin down.

Robust standard errors are typically larger than OLS standard errors, making fewer features appear significant. This is the correct adjustment: OLS was overstating precision by ignoring the correlation structure in the errors.

The econometric toolkit offers further specialized corrections:

• WLS (Weighted Least Squares): explicitly models heteroscedasticity when the variance function is known or estimable
• GLS/FGLS (Generalized / Feasible GLS): corrects for both heteroscedasticity and autocorrelation by transforming the model
• HAC (Newey-West): the standard correction for autocorrelation in a single time series, and the right tool once the panel is collapsed to one series per date — which is exactly what the IC calculation in the next section does
• Fama-MacBeth: estimates the cross-section separately on each date and draws inference from the time series of those estimates, which sidesteps the within-date dependence rather than modelling it

These are valuable tools for the data modeling culture — but they fix inference quality, not prediction quality. A coefficient with correct standard errors still tells you about in-sample relationships, not out-of-sample forecasting power.

Section 4: Why This Doesn't Help Prediction

We now compute the one metric that matters for trading: the out-of-sample Information Coefficient (IC). Because trading decisions rank assets at each decision date, IC is computed cross-sectionally — Spearman rank correlation between predicted and realized returns within each date — and then summarized across the validation period via mean, IR ($\bar{IC} / \sigma_{IC}$), and a t-statistic on the IC time series.

That t-statistic needs the same care Section 2 demanded. Each date's IC is measured against a 21-day forward return, so consecutive daily ICs share 20 of 21 days of outcome window and the series is autocorrelated by construction. Dividing by $\sigma_{IC} / \sqrt{n}$ treats the dates as independent and overstates significance by roughly the square root of the overlap. We report the HAC-corrected statistic with the lag set to the label horizon, and the naive one beside it to show the size of the error.

Validation Predictions and Residuals

The left panel compares predicted vs realized returns; the right panel shows residuals against predictions. We sample for readability on large validation sets.

Why out-of-sample $R^2$ can go negative

Both in-sample and out-of-sample $R^2$ use the same formula:

The denominator $SS_{\text{tot}}$ measures total variance around the mean; the numerator $SS_{\text{res}}$ measures residual variance after the model's predictions. When $R^2 = 0$, the model is no better than predicting $\bar{y}$ for every observation.

In-sample, OLS with an intercept guarantees $R^2 \geq 0$. The intercept ensures that the predicted mean equals the observed mean ($\bar{\hat{y}} = \bar{y}$), so the model can always do at least as well as the constant-mean baseline. The fitted coefficients minimize $SS_{\text{res}}$ by construction.

Out-of-sample, both guarantees break. First, the coefficients were optimized for the training set, not the validation set — they may amplify noise rather than capture signal, making $SS_{\text{res}} > SS_{\text{tot}}$ and driving $R^2$ below zero. Second, the intercept was calibrated to the training-period mean return. If the validation period has a different mean (as it usually does with non-stationary financial data), predictions are systematically shifted to the wrong level. This level error inflates $SS_{\text{res}}$ even if the model ranks returns correctly.

The out-of-sample $R^2$ printed above is negative: the model's magnitude predictions are worse than the naive mean, driven by overfitting 52 coefficients and a stale intercept.

The IC answers a different question, and the two can disagree. IC (Spearman rank correlation) is invariant to level shifts and scaling, so a model whose magnitudes are badly calibrated can still rank correctly — which is why it, not $R^2$, is the standard metric for cross-sectional return prediction in quantitative finance. Read the IC against its HAC p-value rather than its sign: this is one validation fold of one case study, and the naive statistic printed beside it overstates the evidence, because it counts overlapping windows as independent observations.

None of the diagnostic tests — Breusch-Pagan, residual autocorrelation, Jarque-Bera, VIF — measure out-of-sample ranking accuracy. A coefficient can be statistically significant yet contribute nothing to prediction (large sample, tiny effect), and vice versa. The diagnostics tell us whether our inference about parameter values is reliable; they say nothing about whether those parameters produce useful forecasts.

This is the core insight of Section 11.1: the inferential toolkit and the predictive toolkit answer different questions. For algorithmic trading, the relevant question is "does this model rank future returns accurately?" — and the relevant tool is out-of-sample evaluation, not hypothesis testing.

Section 11.2 introduces regularization — the tool that directly targets prediction quality by trading bias for variance.

Key Takeaways

1. Statsmodels provides the full inferential toolkit: standard errors, t-statistics, p-values, F-tests, and a battery of diagnostic tests that scikit-learn intentionally omits.

2. The ETF panel violates the spherical-errors assumption in both of its parts: the error variance depends on the regressors (Breusch-Pagan rejects) and residuals are autocorrelated within each asset, largely because a 21-day forward label makes consecutive observations overlap. Features are also pervasively collinear (many with VIF > 10).

3. A statistic computed on a panel measures what its row order says it measures. Durbin-Watson on a date-ordered panel reports cross-sectional dependence under the name of serial correlation; the same ordering makes Newey-West lag across assets rather than across time. Regroup by asset, pick a covariance estimator that assumes independence only where the diagnostics did not find dependence, and say which one the number came from. Here the overlapping label correlates residuals across symbols and across nearby dates, which rules out clustering on either dimension and leaves Driscoll-Kraay.

4. Robust standard errors repair one failure only. They fix the variance estimate when errors are non-spherical. They cannot rescue a model whose exogeneity or functional form is wrong, where the point estimates themselves are not consistent — and they do not improve predictions.

5. For prediction, we need a different approach: regularization + out-of-sample evaluation. The next notebook introduces Ridge, LASSO, and Elastic Net.

Next: 02_regularization_paths introduces the prediction-oriented framework with regularized regression and walk-forward cross-validation.