AI Weather Models Show No Uniform Deficit at Extremes Across Eleven Systems

The arXiv preprint evaluated deterministic forecasts for wind, temperature, solar radiation and precipitation over ten months at synoptic, solar and rain-gauge stations. Scoring used mean absolute error stratified by ERA5 1991-2020 climatological regimes, with ECMWF IFS as reference. Results showed model-specific rather than class-specific behavior: three Jua models gained 9-11% at P75-P95 precipitation while ECMWF AIFS lost 4.9% in the heat tail and NOAA GFS lost 22.8%. All systems, AI and NWP alike, exhibited shared conditional bias toward the distribution center.

This directly contradicts the narrative that first-generation AI weather models inherently underperform at extremes. The finding matters for disaster preparedness because operational centers are already testing AI systems for high-impact events; blanket skepticism based on early regression models risks discarding tools that improve tail performance for specific variables. Connections to prior work on GraphCast and FourCastNet evaluations are clear: those studies used reanalysis only and deterministic metrics, missing the station-based, regime-stratified design used here.

The next step is operational ensemble verification and longer test periods that include rare compound extremes. Until then, model selection rather than AI-versus-physics framing will determine forecast value for heat, wind and precipitation warnings.