Informal Seminar: Evaluating Extreme Precipitation Forecasts: A Threshold-Weighted, Spatial Verification Framework for AI and NWP Model Comparison

The rapid rise of AI-based weather prediction (AIWP) models has introduced a compelling alternative to traditional NWP models, offering competitive forecast skill at a fraction of the computational cost for many forecast users. However, rigorously evaluating these models, especially high-impact, extreme precipitation events, remains a significant challenge, particularly when spatial coherence matters and models operate at differing grid resolutions. 

This talk presents a verification framework designed to address these challenges. The approach integrates spatial verification methods with proper scoring rules, extending the established High-Resolution Assessment (HiRA) framework through the incorporation of threshold-weighted proper scores. This combination enables user-oriented evaluation that reflects how some operational meteorologists interpret forecasts and how simple post-processing systems may use them in practice. 

A key feature of the framework is its flexibility: by allowing users to assign varying weights to different decision thresholds, it supports targeted evaluation of extreme events or other important decision thresholds.  The framework is applied to evaluate an AIWP model against a high-resolution NWP model, offering insights into how their ability to predict extreme precipitation varies.