18th Workshop on high performance computing in meteorology

Through the Scalability Programme, ECMWF has invested significant resources in the enhancement of efficiency and scalability of the forecasting system - from observational data handling at the beginning of the processing chain, to data assimilation, running the forecast model and dealing with vast amounts of model output data. The recently made case for the next HPC upgrade has demonstrated again the need for substantial efficiency gains in order to meet the strategic target of running a global forecast ensemble at 5 km spatial resolution in 2025.
ECMWF's current research foci aim at two key sources of efficiency in the future: the separation of concerns and the fully integrated and distributed model output data handling.
The separation of concerns aims to create flexibility for algorithmic and numerical choices at the science-code level and highly specialised modules that exploit various processor technology options at the kernel level. In between, options for domain specific languages supported by generic libraries for performing operations on model fields optimizing memory access and parallelism are being developed. This effort is strongly supported by FET-HPC projects like ESCAPE, ESCAPE-2 and EuroEXA, together with many European partners.
The data handling strategy aims to overcome the file oriented and disk based approach and integrates data operations into the pre and post-processing chain through object stores much closer to where data is needed or created. This offers options to exploit new memory layers with less latency and high access bandwidth during model runtime, and flexible work management with distributed processing for model output post-processing. This work is based on partnership projects like NextGenIO and MAESTRO.
The talk will focus on these areas but will also touch on the bigger picture of weather and climate prediction in Europe, that is currently proposed as the European Flagship project ExtremeEarth.