11.4 Trend, Seasonality, Validation, and Forecasting Baselines

Section 11.3 built a regular series and trailing features that obey availability time. We can now ask two different questions: what structures describe the history, and how well can a procedure predict genuinely later values? Decomposition answers the first; walk-forward validation answers the second.

Separate trend, seasonality, and remainder

A time series often contains a slowly changing level, a repeating calendar pattern, and irregular variation. An additive description is

where $T_t$ is trend, $S_t$ is seasonality, and $R_t$ is the remainder. Additive structure assumes seasonal amplitude is roughly constant in target units. If weekly peaks grow as the level grows, a multiplicative description may fit better:

For a positive series, a logarithm turns multiplication into addition, making an additive decomposition on the log scale a useful comparison.

A period counts observations per seasonal cycle. Daily data with a weekly pattern uses period 7; hourly data with a daily pattern uses 24. The number has no calendar meaning until frequency is established. Guessing period 7 on irregular events does not create a weekly model.

STL—Seasonal and Trend decomposition using LOESS—estimates an additive trend, seasonal component, and remainder. Its robust option can reduce the influence of isolated extremes on trend and seasonality:

from statsmodels.tsa.seasonal import STL

development_series = daily.loc[:development_cutoff, "orders"].asfreq("D")
decomposition = STL(
    development_series.interpolate(limit=2),
    period=7,
    robust=True,
).fit()

decomposition.plot()

Interpolation here is an explicit, limited assumption. Keep a coverage indicator and do not bridge long outages. The final test period remains outside decomposition choices if the decomposition informs forecasting features or model selection.

The decomposition lab is a drawing table rather than a parameter panel. Shape the trend with a pencil and draw a reusable seven-phase seasonal stencil; the remainder is recomputed from the additive identity after every stroke. A remaining slope or weekly ripple exposes unfinished structure, while competing low-error sketches show why decomposition is not automatically unique.

A useful remainder has less visible structure, but “looks random” is not proof of independence. Inspect its time plot, distribution, autocorrelation, variance changes, and known events. A residual spike may be a data error, promotion, outage, holiday, or genuine surprise. Verify context before removing it.

Decomposition is descriptive and may use centered smoothers that see both sides of a timestamp. Those trend estimates are not automatically available for real-time prediction. Forecast features must still be generated inside each training window with only past information.

Make every forecast beat a time-valid baseline

A sophisticated forecast is not useful merely because it produces a line. Compare it with simple rules under the same prediction protocol:

• Mean baseline: predict the training-window mean.

• Last-value naive: repeat the most recent available observation.

• Seasonal naive: repeat the value from the corresponding position in the last complete cycle.

• Trailing-mean baseline: predict a recent-history average computed only from values available at origin.

The strongest simple baseline depends on the series. Weekly order demand can make seasonal naive much harder to beat than the global mean.

Forecast protocol must define the origin, horizon, update policy, and gap. A fixed-origin 14-day forecast is made once and cannot update from actual values arriving inside those 14 days. A rolling one-step protocol may update after each true value becomes available. Mixing the two makes results incomparable.

Random train/test splitting is invalid for future forecasting because training may include observations after validation. A walk-forward split keeps every validation window later than its training history. TimeSeriesSplit creates expanding training sets by default and supports test_size, gap, and max_train_size:

from sklearn.model_selection import TimeSeriesSplit

splitter = TimeSeriesSplit(
    n_splits=5,
    test_size=14,
    gap=1,
    max_train_size=180,
)

Comparable duration-based fold metrics require equally spaced samples. If events are irregular, first create a justified regular aggregation or implement cutoffs by timestamps rather than row counts.

The validation lab seals the next seven observations at four successive forecast origins. At each origin, sketch and submit a forecast using only the visible history; only then can the future be revealed and scored against seasonal naive. The submitted fold stays locked, so future truth cannot flow backward into the prediction that it evaluates, and the fold-by-fold scoreboard exposes temporal instability.

Report errors by both fold and horizon. Aggregate MAE answers the typical absolute miss in target units. RMSE emphasizes large misses. MASE divides model MAE by a naive in-sample scale and can support comparison across series when its denominator and seasonal lag are declared. Percentage errors behave poorly near zero, so do not choose them mechanically.

Plot actual and forecast trajectories, residuals over time, error by horizon, and fold distributions. A single average can hide failure after a trend break or only at long horizons. Prediction intervals should be evaluated by empirical coverage and width on later windows, not by visual symmetry.

Once feature definitions, preprocessing, model, hyperparameters, update policy, and baselines are selected from development folds, refit on all development history and evaluate the sealed final period once. Record the exact training cutoff, data revision, timezone, forecast origin, horizon, and retraining schedule so the result can be reproduced.

This chapter closes the course's modeling arc. We learned geometry without labels, compressed it with PCA, constructed calendar-aware features, separated historical structure, and evaluated future prediction without allowing tomorrow to leak into today. The same discipline—define the question, preserve data boundaries, visualize behavior, compare baselines, and state limitations—connects every analysis in the course.