Most forecasters would have taken the easier path: predict the outages already happening. It is a safe bet, a model that looks smart on paper but delivers little real value. The builder of this storm outage model for 216 counties across Missouri and Illinois chose the harder, more honest route. By targeting only *new* outages, those that begin in the six-hour window starting two hours from now, they forced their XGBoost system to earn its keep. The result is a 2.4× improvement in PR-AUC over a baseline built on NWS warnings and SPC outlooks. That is not hype; it is a measurable shift in what a spreadsheet user, or anyone managing operational risk, should expect from their data tools.
This approach mirrors the philosophy behind Designing LLM chat experiences that work for everyone: user outcomes come before technical cleverness. The model does not credit itself for catching storms already knocking out power. It asks, "What is about to happen that we can act on?" That question is the same one any spreadsheet user should ask when building a forecast, whether for inventory, staffing, or budget. The builder also made a crucial decision to use HRRR forecasts as issued, not observed weather, and to train on 2020-2024 data while testing on 2025. That is the difference between a model that works in theory and one that works when the next storm actually arrives. The early transfer test, where the model trained only on Missouri and Illinois caught 31% of new outages in Iowa, Indiana, and Michigan versus 19% for the baseline, suggests this approach generalizes beyond its training region. For anyone managing multi-state operations, that is a concrete reason to explore similar methods.
The precision of 15% at equal alert volume sounds low, but context matters. The base rate is 0.5% of county-hours, outages are rare events. Catching 30% of new outages with that precision is not a vanity metric; it is a practical tool for prioritization. Compare this to Three quick checks that catch hidden errors in bank statement imports, where small process changes catch quietly wrong data. Here, the quiet wrongness would be pretending a model that predicts ongoing outages is useful. This builder avoided that trap by designing a target that is genuinely hard to beat. The live forecast has been running since November 1, with every prediction saved for blind testing. That commitment to honest evaluation is rare and worth watching.
One detail to track: the model uses neighboring county current outages as a feature. That introduces a dependency that could break down in widespread events where every county is already affected. The builder has acknowledged this in their leakage audit. The open question is whether the model's performance holds when the grid is under maximum stress. The answer will tell us if this is a breakthrough or a promising prototype. For now, the takeaway is clear: define your problem honestly, test on unseen future data, and let the metrics speak for themselves. That is how you build forecasts that actually change decisions.