Where academic tradition
meets the exciting future

You are here: TUCS > PUBLICATIONS > Publication Search > On Policymakers’ Loss Functi...

On Policymakers’ Loss Functions and the Evaluation of Early Warning Systems

Peter Sarlin, On Policymakers’ Loss Functions and the Evaluation of Early Warning Systems. Economics Letters 119(1), 1–7, 2013.

http://dx.doi.org/10.1016/j.econlet.2012.12.030

Abstract:

This paper introduces a new loss function and Usefulness measure for evaluating early warning systems (EWSs) that incorporate policymakers’ preferences between issuing false alarms and missing crises, and individual observations. The novelty derives from three enhancements: (i) accounting for unconditional probabilities of the classes, (ii) computing the proportion of available Usefulness that the model captures, and (iii) weighting observations by their importance for the policymaker. The proposed measures are model free such that they can be used to assess early warning signals issued by any type of EWS, and flexible for any type of crisis. Applications to two renowned EWSs, and comparisons to two common evaluation measures, illustrate the importance of an objective criterion for choosing a final specification and threshold value, and for models to be useful, the need to be more concerned about the rare class and the importance of correctly classifying observations of most relevant entities.

BibTeX entry:

@ARTICLE{jSarlin_Peter13b,
  title = {On Policymakers’ Loss Functions and the Evaluation of Early Warning Systems},
  author = {Sarlin, Peter},
  journal = {Economics Letters},
  volume = {119},
  number = {1},
  pages = {1–7},
  year = {2013},
  keywords = {Early warning systems, Policymakers’ loss functions, Policymakers’ preferences, Misclassification costs},
}

Belongs to TUCS Research Unit(s): Data Mining and Knowledge Management Laboratory

Publication Forum rating of this publication: level 2

Edit publication