Workshop: Building reproducible workflows for earth sciences

From loose scripts to ad-hoc reproducible workflows: a methodology using ECMWF's ecflow

Speaker

Damien Decremer (ECMWF)

Description

With a rising need for resource-efficient and fully tractable software stacks, e.g. in operational contexts, a programmer increasingly faces ever-changing complex software- and hardware dependencies within their workflow. In this talk, we show how ecflow was used to adaptively build up a fully reproducible workflow at runtime for the calibration of the European Flood Awareness System's hydrologic model, although the presented methods can be adapted to any workflow needs. Whether we require serial, parallel, or mixed-mode executions on a single or multiple computer systems, ecflow can help to easily manage, monitor and adapt any software stack without penalising execution performance.

Primary author

Damien Decremer (ECMWF)

Co-author

Cinzia Mazzetti (ECMWF)

Presentation materials

#repwork19