18th Workshop on high performance computing in meteorology

OMNI/O is a new tool being developed to accurately record, analyze and replay file input and output performed by parallel applications without modification to the applications being profiled. The tool consists of four components which in their entirety enable the collection and replay of I/O patterns and statistics for common forecast models and data assimilation systems. The “trace” component records events for each thread and processor by intercepting operating system calls and logging the necessary information to replay all events. The recorded events can be analyzed using the “statistics” component to gain insight into an applications I/O pattern, throughput and opportunity for optimizations. The “preproc” component takes the recorded events and prepositions real or fake data files at user configurable locations to allow for characterization of different file systems using the same pattern as the profiled application. The final component, “replay”, executes the desired I/O pattern without the original profiled application being present. It is envisioned these representative workloads can then be used for procurement benchmarks, to motivate additional scenarios, and to understand limitations in current systems. This presentation will focus on the techniques used to develop the tool along with results from its application on common forecast models such as HRRR and FV3 coupled to the GSI data assimilation system.