financial modeling

Time-series AI models stumble against the simple baseline in market forecasting

It's refreshing to see someone put time-series foundation models to a real, honest test.

3 min readMachine Learning
Time-series AI models stumble against the simple baseline in market forecasting
I benchmarked TimesFM 3.0, Chronos-2 and Toto 2.0 on market time series. None beat simple baseline. [P]

A simple baseline just beat three of the most talked-about time-series AI models at stock market forecasting, and the results are a useful reality check for anyone who works with data. A developer ran zero-shot experiments using Google's TimesFM 3.0, Amazon's Chronos-2, and Datadog's Toto 2.0 on 24 US stocks and ETFs over 504 trading days, then compared their predictions to a naive strategy: simply predicting the market would go up every day. That naive approach achieved 52.2% directional accuracy. The best AI model managed 51.1%. The models performed about as well on simulated random walks, data with zero predictive signal, as they did on real stock prices. When tested in a simple long-or-cash trading simulation, buy-and-hold returned $15,036 on a $10,000 starting stake. The best model-driven strategy returned $12,181.

This result echoes patterns we have seen elsewhere in applied forecasting. Our coverage of Four Time-Series Models Share a Core Architecture, Diverge on Data Sharing showed how these foundation models share a common structural approach, scaling, patching, and predicting quantiles, while diverging mainly in how they share information across series. The critical difference is that those models were evaluated on general-purpose forecasting tasks with stable patterns, not on financial markets where the signal-to-noise ratio is notoriously low. Our piece on Forecasting county power outages two hours ahead with AI accuracy demonstrated that AI can deliver real value when applied to problems with measurable, repeatable patterns. Power outages follow physical constraints and historical recurrence; stock prices follow the collective decisions of millions of participants who are themselves trying to predict each other.

The practical takeaway is direct: do not assume that impressive general-purpose performance transfers to every domain. These models are powerful tools for forecasting electricity demand, inventory levels, or website traffic, problems where the past meaningfully constrains the future. Financial markets are a fundamentally different class of problem because predictive signals, if they exist, are quickly arbitraged away. The developer who ran these tests fed the models 512 historical closing prices and asked for predictions one, five, and twenty days ahead. That is a perfectly reasonable setup for a retail price forecast. It is a terrible setup for stock market prediction, where the efficient market hypothesis suggests that all publicly available information is already priced in. The models did not fail because they are bad at time-series forecasting. They failed because the task itself, in this configuration, may be unsolvable.

What matters now is how practitioners interpret these results. The temptation will be to dismiss the models entirely, but that would be a mistake. The smarter response is to ask where the boundary lies between a solvable forecasting problem and an adversarial one. That boundary is exactly what our coverage of When Your Reproduction Doesn't Match the Published Results explored, the gap between published performance and real-world replication is often not a bug in the model, but a mismatch between the evaluation context and the deployment context. For anyone building AI-native tools, the open question is not whether these models can predict stock prices, but whether we are asking them the right question in the first place.

From Machine Learning

I may be naive, but I've been curious whether the recent wave of time-series foundation models can actually extract useful predictive signals from financial markets.

Models like Google's TimesFM 3.0, Amazon's Chronos-2, and Datadog's Toto 2.0 are impressive general-purpose forecasters from the first sight. But stock prices are difficult to predict, so I wanted to see how these models perform against very simple baselines.

Read the original at Machine Learning