Businesses and organisations who provide services to customers know the importance of job schedulers which orchestrate the execution and facilitate the integration of processes that run on different computers. Such job scheduling, or workload automation, provides a point of control to monitor unattended computer programs.
ecFlow is the workflow package, developed at ECMWF, that enables users to run a large number of programs (with dependencies on each other and on time) in a controlled environment. It provides reasonable tolerance for hardware and software failures, combined with good restart capabilities. It is used at ECMWF for most of our operational suites, across a range of platforms.
ecFlow executes tasks (jobs) and receives acknowledgements from the associated job when it changes status or when it sends events. It does this using child commands embedded in the scripts.
ecFlow stores the relationship between tasks and is able to submit tasks dependent on triggers.
ecFlow is complemented by ecflow_ui, its graphical interface that allows users to have immediate grasp, using colour coding, on the status of the various programs or processes handled by the scheduler.
The course will look at the various elements of the scheduler and its graphical interface. By the end of the course participants will have a good understanding of the various features of ecFlow and be capable of designing their own suite.
Requirements
The practical sessions for all modules will involve using a UNIX interactive system. Participants of any module must know basic UNIX commands, including how to use an editor (e.g. vi, etc.). Moreover, some knowledge of Python would be useful.
Participants are expected to work on a project independently (we recommend participants to come with their own project to develop during the course).
All the lectures and practical activities will be in English.