Virtual Event: ECMWF/EUMETSAT NWP SAF Workshop on the treatment of random and systematic errors in satellite data assimilation for NWP

Europe/London
ECMWF

ECMWF

Virtual
Description

Workshop description

Dealing with random and systematic errors in observations and models is at the heart of satellite data assimilation for Numerical Weather Prediction (NWP). This workshop captured recent progress and the state of the art in this area, identified key issues and limitations, and discussed avenues for further developments. It was organised in collaboration with the EUMETSAT NWP SAF.

The workshop covered the following main themes:

1) Estimating uncertainties

Knowledge about observational and model errors and uncertainties informs our treatment of these in the assimilation. This theme covered recent activities ranging from thorough characterisation of satellite instruments, better estimation and modelling of other error contributions (such as radiative transfer/observation operator errors or other representation errors), the use of reference observations, to advances in diagnostic methods.

2) Correction of observational and model biases in data assimilation

Correction of observational biases is essential for the successful assimilation of many satellite observations. Adaptive bias correction methods are now commonly used, but separation of observation and forecast model biases continues to be challenging. This is particularly true in our present era with an ever increasing number of observations needing bias correction, but also in systems with highly variable observational coverage such as reanalysis. In addition, estimation of model bias during the assimilation is also becoming a reality, further probing our ability to separate different sources of errors and biases. This theme covered practical experiences with these issues, including approaches to introduce further prior knowledge to constrain or aid the separation of different sources of bias.

3) Representing observation errors in data assimilation

The description of the random component of observation error is key to optimising the impact of any observation, and this area has seen considerable progress in recent years. Many satellite observations are strongly affected by representation error, and this is often situation-dependent and correlated between different observations. This is now increasingly taken explicitly into account, with the treatment of inter-channel error correlations for satellite radiances, for instance, becoming wide-spread. The treatment of spatially and temporally correlated observation errors is also emerging. This poses questions not only in terms of how to estimate and specify such correlations, but also how to account for them in a computationally affordable way. This theme covered recent advances in the more complete specification of observation errors, including technical as well as conceptual challenges.

While the primary application focus of the workshop was Numerical Weather Prediction (NWP), the way in which these issues impact climate re-analyses based on NWP systems were also be examined.

Workshop aims

The aim of this workshop was to capture the state of the art in the treatment of random and systematic errors in satellite data assimilation for NWP, to identify key issues and limitations, and to identify avenues for further progress. Recent developments and open issues were covered through oral and poster presentations, and targeted working groups were established to identify and recommend future research needs. The output will be in the form of working group reports, identifying key recommendations for NWP centres.

 

Registration
ECMWF/EUMETSAT NWP SAF Workshop on the treatment of random and systematic errors in satellite data assimilation for NWP
Events team
    • 11:00 13:40
      Session 1: Welcome and scene-setting talks
      Convener: Patrick Laloyaux (ECMWF)
      • 11:00
        Welcome 10m
        Speaker: Andy Brown (ECMWF)
      • 11:10
        Workshop logistics 5m
        Speaker: Patrick Laloyaux (ECMWF)
      • 11:15
        A review of the evolution of setting observation errors in satellite DA 40m
        Speaker: Niels Bormann (ECMWF)
      • 12:00
        Lunch break 1h
      • 13:00
        Disentangling biases in observations and models over the years 40m
        Speaker: Dick Dee (JCSDA)
    • 13:45 16:10
      Session 2: Estimating uncertainty
      Convener: Bill Bell (ECMWF)
      • 13:45
        Metrology-inspired approaches to characterisation of uncertainty in Earth observations 25m
        Speaker: Chris Merchant (University of Reading)
      • 14:15
        Feedback between GSICS and the NWP Community on Radiometric and Spectral Biases 25m
        Speaker: Tim Hewison (EUMETSAT)
      • 14:45
        Coffee break 30m
      • 15:15
        Characterization of Cross-track Infrared Sounder calibration uncertainties and Noise behavior 25m
        Speaker: David Tobin (SSEC)
      • 15:45
        Use of uncertainty information from reference observations in model and satellite cal/val 25m
        Speaker: Stuart Newman (Met Office)
    • 16:15 17:15
      Poster session
    • 17:15 17:45
      Catch up with Monday's speakers: Chris Merchant, David Tobin, Stuart Newman, Dick Dee, Tim Hewison, Niels Bormann
    • 09:00 11:00
      Session 2 continued: Estimating uncertainty
      Convener: Niels Bormann (ECMWF)
    • 11:00 13:30
      Session 3: Correction of model and observational biases
      Convener: Alan Geer (ECMWF)
      • 11:00
        Estimation of model biases and the importance of scale separation 25m
        Speaker: Patrick Laloyaux (ECMWF)
      • 11:30
        Exploring Model Error with Machine Learning 25m
        Speaker: Massimo Bonavita (ECMWF)
      • 12:00
        Lunch break 1h
      • 13:00
        The role of GNSS radio occultation measurements constraining biases in NWP and reanalyses 25m
        Speaker: Sean Healy (ECMWF)
    • 13:30 17:15
      Session 3 continued: Correction of model and observation biases
      Convener: David Duncan (ECMWF)
      • 13:30
        Bias correction of observations based on an analysis that uses only anchor observations 25m
        Speaker: Mark Buehner (Environment and Climate Change Canada)
      • 14:00
        The treatment of biases in the ERA5 global reanalysis 25m
        Speaker: Bill Bell (ECMWF)
      • 14:30
        Coffee break 30m
      • 15:00
        Understanding the role of analysis error in convergence of reanalysis production streams in MERRA-2 25m
        Speaker: Amal Elakkraoui (NASA)
      • 15:30
        Constrained variational bias correction 25m
        Speaker: Wei Han (JCSDA)
      • 16:00
        Break 15m
      • 16:15
        Panel discussion on treating biases in satellite DA (Moderator: Bill Bell) (Panellists: Hans Hersbach, Robin Faulwetter, Sergey Frolov, Tony McNally) 1h
    • 17:15 17:45
      Catch up with Tuesday's speakers: Bill Bell, Amal Elakkraoui, Paul Poli, Patrick Laloyaux, Massimo Bonavita, Sean Healy, Mark Buehner, Wei Han, Michael Rennie, Xavier Calbet
    • 09:00 11:00
      Session 4: Observation errors
      Convener: Massimo Bonavita (ECMWF)
      • 09:00
        Error correlations and hyperspectral sounders 25m
        Speaker: Fiona Smith (Bureau of Meteorology - BoM)
      • 09:30
        Including the horizontal observation error correlation in the assimilation of AMSU-A data 25m
        Speaker: Koji Terasaki (RIKEN)
      • 10:00
        Estimating observation errors: diagnostics, possibilities, and pitfalls 25m
        Speaker: Sarah Dance (University of Reading)
      • 10:30
        Coffee break 30m
    • 11:00 15:45
      Session 4 continued: Observation errors
      Convener: Kirsti Salonen (ECMWF)
      • 11:00
        Treating uncertainties in the assimilation of AMVs 25m
        Speaker: James Cotton (Met Office)
      • 11:30
        Situation-dependent inter-channel error correlations for all-sky data assimilation 25m

        Satellite observations sensitive to cloud and precipitation are assimilated in operational weather forecasting using the `all-sky' approach. A key part of this is a situation-dependent model for observation error, which inflates the variances in cloudy conditions to represent large increases in model error and representation error. Observation error correlations are in general not yet represented, despite the fact that model error and representation error can generate strong error correlations between channels and between observations. To complicate the picture there are state-dependent biases associated with inadequate modelling of cloud and precipitation in forecast models. Further complication comes from the importance of variational quality control in all-sky assimilation, and the potential difficulty of combining this with the error correlations. This work explores two ways to account for situation-dependent error correlations. One is to use a single all-sky error covariance matrix with eigenvalue scaling to provide situation-dependence. This relies on the projection of most of the cloud signal onto the leading eigenvector of the error covariance matrix. The second approach selects from more than 100 situation-dependent error covariance matrices stored in a lookup table. This has advantages particularly for all-sky microwave assimilation, where channel usage is variable and cloud projects onto several of the eigenvectors. In both methods, systematic error patterns can be amplified in an undesirable way by the trailing eigenvectors, leading to degradation of the analysis. This can be addressed through reconditioning, and to understand such features of error covariance modelling, the "eigenjacobians" and "eigendepartures" proved to be useful tools. Although some issues remain, both the eigenvalue scaling and the lookup table approaches are viable ways of representing situation-dependent inter-channel error correlations.

        Speaker: Alan Geer (ECMWF)
      • 12:00
        Lunch break 1h
      • 13:00
        Pragmatic approaches for treating spatial error correlations: thinning, inflation, gradient observations 25m
        Speaker: Joël Bédard (Environment and Climate Change Canada)
      • 13:30
        A method for representing spatially correlated observation errors for wind data 25m
        Speaker: Oliver Guillet (Météo-France)
      • 14:00
        Coffee break 30m
      • 14:30
        Panel discussion on observation errors (Moderator: Niels Bormann) (Panellists: Jo Waller, Olaf Stiller, Peter Weston, Pierre Gauthier, Yann Michel) 1h
      • 15:30
        Introduction to Working Groups 15m
        Speaker: Bormann Niels (ECMWF)
    • 15:45 16:45
      Catch up with Wednesday's speakers: Sarah Dance, Mary Forsythe, Joël Bédard, Oliver Guillet
    • 09:00 10:00
      Poster session and catch up with speakers (Koji Terasaki, Fiona Smith)
    • 10:00 16:30
      Working groups
      • 10:00
        WG1: Treatment of biases I - chaired by Paul Poli (EUMETSAT) and Patrick Laloyaux (ECMWF) 5h
      • 10:00
        WG2: Treatment of biases II - chaired by Stuart Newman (Met Office) and Sean Healy (ECMWF) 5h
      • 10:00
        WG3: Treatment of observation errors I - chaired by Nancy Nichols (University of Reading) and Peter Weston (ECMWF) 5h
      • 10:00
        WG4: Treatment of observation errors II - chaired by Jo Waller (Met Office) and Stephen English (ECMWF) 5h
      • 15:00
        WG chairs finalise reports 30m
      • 15:30
        Plenary and close 1h
        Speaker: Niels Bormann (ECMWF)