A forecasting atlas where a one-line baseline beats a foundation model
Published:
A forecasting atlas where a one-line baseline beats a foundation model
ChronoScope is a univariate time-series forecasting atlas: fifteen diagnostic cases, four of them real and licensed (UCI Electricity, UCI Beijing PM2.5, Monash/M4 hourly and daily), each run through the same nineteen-method ladder and scored by backtested MASE, sMAPE and interval coverage. The ladder goes from classical baselines up through gradient boosting and deep nets to a foundation-model tier: Chronos-Bolt, Chronos-2, TimesFM-2.5 and TiRex-2, all zero-shot.
The reason I built it as an atlas rather than a leaderboard is the result you only see when everything runs on the same footing:
TiRex-2 0.476 (a foundation model earns its place here)
SeasonalNaive 0.641
TimesFM-2.5 0.729
A one-line SeasonalNaive beats a foundation model on this real series. White-noise and random-walk controls sit on the board too, so a method that "wins" on noise is visible as such.
The foundation models are real and they run, but offline: the checkpoints are heavy and one of them (TiRex-2) needs a WSL2/CUDA lane because its kernels have no Windows wheels, so the browser lane is a numpy core and the foundation-model numbers are baked. Zero-shot forecasting is genuinely useful, and sometimes it is the best thing on the board. It is also, sometimes, beaten by a method you can write in one line, and I would rather show both. Live · source.