Time series · Spectral analysis · Box-Jenkins

SPECTRAL_ARIMA

Use the spectrum to guide model discovery—and show how the model was chosen.

Syntax

SPECTRAL_ARIMA(series)
SPECTRAL_ARIMA(series,period)
SPECTRAL_ARIMA(series,period,maxModels)
SPECTRAL_ARIMA(series,period,maxModels,horizon)

Period 0 means automatic detection. Explicit periods are integers 2–4096, must span at least three training cycles, and must fit at least two cycles in the Welch window. Discovery is limited to 4–64 candidate fits (default 24). Horizon is 1–1000 periods (default 12). Validation uses max(12,horizon), capped at one quarter of retained observations.

Examples

water=NDBC("42040","WTMP",2000,2025)
model=SPECTRAL_ARIMA(water)
EXTRACT(model,"slideshow")
model=SPECTRAL_ARIMA(water,365,24,30)
EXTRACT(model,"coverage")
EXTRACT(model,"candidates")
EXTRACT(model,"residuals")

A specified 365-day annual cycle is evaluated with a 365.2425-day harmonic period on daily data. Monthly annual seasonality uses 12; weekly annual harmonics use 365.2425/7 when period 52 is specified.

How selection works

  1. Validate calendar order and split at gaps/nonfinite observations; retain every finite observation with explicit block boundaries.
  2. Reserve a validation tail in the latest block with at least 64 observations; later short blocks are withheld until final refitting. Detrend blocks separately and pool within-block Hann-window Welch spectra.
  3. Screen d using an intercept-only augmented Dickey-Fuller regression with an approximate fixed cutoff. Generate nonseasonal and, when a strong period is detected, seasonal or harmonic families.
  4. Fit common coefficients to all eligible training blocks, resetting state at boundaries. Stationary/invertible factors use bounded reflection coefficients; individual AR coefficients may exceed one. Fit low-order seeds, then within-block ACF/PACF-guided proposals. Expand around family winners when residual checks fail and budget remains.
  5. Compare approximate common-tail CSS-AICc only within identical differencing/seasonality/harmonic families. Choose among family winners by the same held-out level forecast RMSE, then refit once across all blocks. Harmonic phase uses actual calendar positions, including missing months/days.

Automatic periods require a peak-to-median-power ratio of at least 8 and a two-bin local prominence of at least 4; these are heuristics, not significance tests. Only the strongest accepted period is modeled. This is a bounded guided search, not an exhaustive order search or Whittle-likelihood estimator.

Orders and resource limits

Nonseasonal p and q are limited to 0–3 with at most five total AR/MA terms; d is 0 or 1 from the screen. For supported short seasons (period at most 24), seasonal AR(1), MA(1) and seasonal-difference-plus-MA(1) families are considered. Long seasons use K=1 or 2 sine/cosine pairs followed by short-memory ARIMA. Harmonic regressors and ARIMA errors are estimated sequentially, not jointly.

Search is sequential. The 60-second discovery budget is checked between fits and during optimizer objective evaluations. Each discovery fit has at most 6,000 evaluations and a 15-second sub-budget; final refitting has at most 12,000 evaluations and 30 seconds. A final refit is extra. FFTWrapper controls Java/native selection; speed depends on data, model orders and backend.

Gaps, forecasts and limits

No interpolation or calendar compression is performed. If gaps split the history, all finite blocks are retained. At least one block of 64 observations is required for spectral screening and validation. Per-block differencing and initialization remove warm-up rows from the innovation objective; coverage reports their counts. Business-day observations are not silently treated as a complete daily calendar; aggregate to a suitable frequency first.

Actual/fitted values are on the original level scale. Initial fitting values are omitted until differencing and CSS initialization finish. Forecast extracts start at the end of the latest block. If it is too short to initialize the selected model, an explanatory report replaces the forecast; an older forecast origin is never substituted. EXTRACT(model,"forecast_chart") shows the previous 12 retained observations plus a shaded ±1 forecast-error SD band. This is approximately 68% pointwise coverage under Gaussian innovations, conditional on fitted parameters. SD propagates AR/MA, seasonal factors and differencing; forecast_sd, forecast_lower and forecast_upper return dated numeric series. Parameter, harmonic-coefficient and model-selection uncertainty are excluded. The forecast extract remains the numeric point forecast.

The new segmented optimizer uses convergence tolerances and explicit evaluation/time limits; failure is reported in candidates. Stability is enforced using reflection coefficients instead of individual coefficient clipping. It does not change the existing SeasonalARIMA class. A low gap-adjusted approximate portmanteau p-value or unresolved seasonal pattern is grounds for revision. One holdout split is not rolling validation.

Time series · Identification · Estimation · Diagnostic checking

From a spectrum to an ARIMA model

The Box-Jenkins approach is an iterative process: identify a plausible model, estimate it, check its residuals, and revise it when unexplained structure remains. A spectrum provides another view of the same serial dependence represented by autocovariances.

What a spectral peak tells us

A peak near frequency 1/12 suggests a cycle lasting about 12 observations. On a monthly calendar, that is annual seasonality. It does not establish p, q, seasonal differencing, or a causal explanation. A deterministic seasonal cycle and a stochastic seasonal process can both produce strong peaks.

AR, MA and integration

AR coefficients describe dependence on past observations; MA coefficients describe dependence on past innovations. For a stationary ARMA process, the two-sided spectrum at angular frequency omega is f(omega) = sigma²/(2*pi) * |theta(exp(-i*omega))|² / |phi(exp(-i*omega))|². The AR and MA lag polynomials therefore determine the theoretical spectral shape. The spectrum can suggest promising structures, while ACF/PACF, estimation and residual checking distinguish competing models. Differencing addresses nonstationarity; it should not be applied merely because a cycle is visible.

Why long seasons use harmonic terms

A sine/cosine pair represents amplitude and phase at a selected period. Additional pairs represent harmonics. This command fits these calendar regressors first and ARIMA to their residuals. This is a sequential approximation to harmonic regression with ARIMA errors, not a joint maximum-likelihood fit. For a daily annual period, it avoids an expensive 365-lag seasonal grid.

When discovery has not finished the job

Residual autocorrelation, poor held-out forecasts, unstable coefficients or unresolved seasonal peaks call for revision. A small score is not evidence of causation, and model-selection uncertainty remains. Missing calendar periods are not adjacent observations: this implementation pools within-block innovations under shared coefficients, resets state at each gap, and never lets an FFT window or residual lag pair cross a gap. Partial-autocorrelation parameters guarantee stable factors while allowing individual AR coefficients above one. Shared coefficients assume a common mechanism across deployments; changes in habitat or instruments can violate that assumption.

Further reading

Box, Jenkins, Reinsel and Ljung, Time Series Analysis: Forecasting and Control, fifth edition (Wiley, 2015). Earlier editions have different author lists, but develop the identification-estimation-diagnostic-checking approach. The explanation here is a paraphrase; no edition-specific page references or reproduced text are assumed.

Hyndman and Athanasopoulos: ARIMA models · Dynamic harmonic regression

Available extracts

Save the command result in a variable, then use EXTRACT(result, "key"). Availability can depend on the input form and fitted model.

In indexed patterns such as beta:n, replace n with the required index.