A discussion paper by Lia Adoha and Bruce Edmonds, entitled “Some pitfalls to beware when applying
models to issues of policy relevance” — a cut-down version of the paper in the Simulating Social Complexity Handbook
Some pitfalls to beware when applying models to issues of policy relevance
Lia Adoha and Bruce Edmonds
Abstract
This paper looks at some of the ways things can go wrong when mathematical or computational
models are applied to inform policy on important issues. It looks at some of the pitfalls in the model
construction and development phase, including: choosing assumptions, the effect of “theoretical
spectacles”, over-simplified models, not understanding model limitations, and not testing a model
enough. It then goes on to discuss the pitfalls that can occur when a model is applied to inform
policy, including: entrenched policies based on models with little or no evidential support, and how
models can narrow the evidential base considered. It also looks at: confusions concerning model
purpose and kinds of question they may answer, when models are used out of context, asking
unreasonable things of models, when the uncertainties are too great, when models give a false
sense of security, and when the focus should be on values rather than facts. This discussion is then
illustrated with two examples, one economic and one from fisheries. It concludes that most of these
problems stem from the interface between the modelling and policy worlds. It ends with some
simple recommendations to reduce these mistakes.
This is a reduced version of:
Aodha, L. and Edmonds, B. (2017) Some pitfalls to beware when applying models to issues of policy relevance. In Edmonds, B. & Meyer, R. (eds.) Simulating Social Complexity – a handbook, 2nd edition. Springer, 801-822. https://link.springer.com/chapter/10.1007%2F978-3-319-66948-9_29