M3R Integrates NEXRAD Radar and PWS Data via Multimodal Attention for Rainfall Nowcasting
Meteorology-informed multimodal AI improves localized rainfall nowcasting for disaster mitigation.
M3R combines visual NEXRAD radar imagery with numerical Personal Weather Station measurements through a temporal alignment pipeline for heterogeneous meteorological data. Weather station time series serve as queries in specialized multimodal attention mechanisms to selectively attend to spatial radar features for precipitation signatures. Experiments on three 100 km x 100 km areas centered at NEXRAD stations show improved accuracy, efficiency, and precipitation detection over prior methods (arXiv:2604.15377).
MetNet previously fused radar and satellite imagery for precipitation forecasting using convolutional LSTM architectures (arXiv:2003.12140). DeepMind's 2021 Nature paper demonstrated skillful nowcasting up to 90 minutes ahead via deep generative models applied to radar alone (https://www.nature.com/articles/s41586-021-03854-z). M3R extends these by incorporating ground station queries and open-sources its code.
The architecture establishes new benchmarks for multimedia-based nowcasting by addressing limitations in single-modality deep learning approaches for direct rainfall prediction and temporal alignment.
AXIOM: M3R shows how attention-driven fusion of radar imagery and ground sensor readings produces more accurate short-term rain forecasts at local scales than radar-only systems.
Sources (3)
- [1]Primary Source(https://arxiv.org/abs/2604.15377)
- [2]MetNet: A Neural Weather Model(https://arxiv.org/abs/2003.12140)
- [3]Skillful Precipitation Nowcasting Using Deep Generative Models of Radar(https://www.nature.com/articles/s41586-021-03854-z)