Virtual Event: ECMWF-ESA Workshop on Machine Learning for Earth System Observation and Prediction

Neural networks (NNs) have emerged as a promising tool in many meteorological applications. While they perform amazingly well at many complex tasks neural networks are generally treated as a black box, i.e. it is typically considered too difficult a task to understand how they work. However, a better understanding of neural networks would have many advantages. A better understanding could provide important information for the design and improvement of NNs, increase trust in NN methods especially for operational use, and even enable us to gain new scientific knowledge from NN models. Fortunately, progress in the computer science field of Explainable AI (XAI) is yielding many new methods that can help scientists gain a better understanding of a NN's inner workings. For example, neural network visualization methods, such as Layer-Wise Relevance propagation (LRP), can help meteorologists extract strategies the neural network uses to make its decisions. Furthermore, viewing the problem more from a meteorologist’s perspective, another important tool is synthetic experiments, where we design synthetic inputs that represent specific meteorological scenarios and test the response of the neural network to those inputs. We present some of these techniques and demonstrate their utility for sample applications. For example, we show how these methods can be used to identify strategies used by a neural network trained to emulate radar imagery from GOES satellite imagery. Finally, we look at the process of gaining insights into neural networks for meteorological applications as a whole – and highlight that it is an iterative, scientist-driven discovery process, that incorporates old fashioned methods of hypothesis generation and testing and experimental design. In this context NN visualization tools simply provide additional tools to assist the meteorologist in this endeavor, and are comparable to a biologist's use of a microscope as tool for scientific analysis.