LASSO_REGRESSION

What does lasso do?

Ordinary least squares chooses coefficients to minimize squared residuals. Lasso adds a penalty for the absolute sizes of the slopes. It can set coefficients exactly to zero, leaving a smaller model. Shrinkage accepts some training-fit bias in pursuit of more stable predictions; improvement on new data is not guaranteed.

Syntax

LASSO_REGRESSION(LIST(Y,X1,...)[,lambda[,maxIterations[,tolerance]]])

The equation and the units

Each predictor becomes Z_ij = (X_ij − mean(X_j)) / s_j, where s_j = √[Σ(X_ij − mean(X_j))²/n]. Y is centered, but is not divided by its standard deviation. The intercept is unpenalized. The displayed slopes are β_j = θ_j/s_j, and the intercept is mean(Y) − Σ β_jmean(X_j).

Standardization gives predictors in different units comparable penalties. Lambda is in response units under this convention. Multiplying Y by 100 requires multiplying lambda by 100 for equivalent shrinkage. Published packages may use different objective scaling.

How the iteration works

1. Center Y, standardize X and start with zero slopes.
2. For one predictor, calculate its association with the residual excluding its own contribution.
3. Soft-threshold that association: S(c,λ) = sign(c) max(|c| − λ,0), divided by the standardized column's mean square.
4. Update the residuals and cycle through all predictors.
5. Stop when the maximum violation of the optimality (KKT) conditions is at most tolerance × max(1,lambdaMax).

This is coordinate descent, not repeated unpenalized OLS. Nonconvergence produces an explicit error instead of a misleading completed fit.

Choosing lambda

Lambda zero removes the penalty. Lambda at or above lambdaMax = max |Z′(Y−mean(Y))| / n yields zero slopes. LambdaMax is reported in the summary. Constants have zero slopes; if every predictor is constant, lambdaMax is zero.

Omit lambda to search automatically using five expanding training windows and disjoint future validation blocks. At least 30 aligned observations are required. The first window uses half the observations. Each window fits its own scaling and lambdaMax using training rows only. Search 30 log-spaced fractions from 1 to 0.001, plus zero; select the smallest pooled validation MSE, with exact ties favoring the larger fraction. Refit all data using the selected fraction times full-sample lambdaMax. A failed candidate stops the search with an explicit error; it is not silently ignored. For time series, use chronological or rolling validation, fitting all scaling on each training window. The example's 0.01 is illustrative, not an optimized value. Do not choose lambda by maximizing training R².

The validation score used for selection is not an independent final test. Reserve an untouched later period for final assessment. Validation uses observed predictors, so it measures conditional prediction rather than forecasting unavailable future inputs. Irregular gaps and any frequency conversion by alignment still require care.

Automatic example

fit=LASSO_REGRESSION(LIST(LOGDIFF(HOUSTNSA),LOGDIFF(PERMITNSA)))
fit

Run with explicit lambda

housing=LOGDIFF(HOUSTNSA)
permits=LOGDIFF(PERMITNSA)
fit=LASSO_REGRESSION(LIST(housing,permits),0.01)
fit

One predictor makes shrinkage easy to see. Add more candidate series to explore variable selection.

EXTRACT(fit,"summary")
EXTRACT(fit,"model")
EXTRACT(fit,"slideshow")

Extractable results

Lasso and an OLS refit answer different questions

The lasso fitted values use the shrunk coefficients. An OLS refit estimates the selected slopes again without a penalty. It is a different model. Its standard errors and p-values do not account for selection; extracting HAC does not resolve that problem.

baseline=EXTRACT(fit,"regression")
refitted=EXTRACT(fit,"refit")
EXTRACT(refitted,"model")

OLS comparisons are evaluated only when requested and can fail for singular designs or insufficient residual degrees of freedom. When all slopes are zero, refit returns an explanation; the lasso model and fitted series already provide the intercept-only result.

Data preparation and interpretation

The existing getSameDateRange alignment runs once and may convert frequency. Inspect the data extract. Remaining nonfinite values, mismatched dates or dates not in strictly increasing order are rejected. Inputs are not deliberately modified. Constant predictors are excluded. The solver permits more predictors than observations and collinear designs, but highly correlated predictors may exchange roles and coefficient solutions need not be unique.

A zero is not a significance test. Retention does not establish causality. Lasso does not remove serial correlation; DW and ACF use adjacent retained observations, so irregular gaps matter. No ordinary OLS p-values, adjusted R², AIC or BIC are attached to the lasso fit.

Ordinary lasso penalizes sine and cosine separately. It is not group lasso and does not preserve harmonic pairs automatically.

Further reading

Friedman, Hastie and Tibshirani (2010), Regularization Paths for Generalized Linear Models via Coordinate Descent. See also Lasso overview.