by Mike Bithell, Edmund Chattoe-Brown and Bruce Edmonds. A paper accepted for Social Simulation 2021
Related resources (slides, references etc.) collected together at http://cfpm.org/model-comparison/
Abstract. Vital to the increased rigour (and hence reliability) of Agent-based modelling are various kinds of model comparison. The reproduction of simulations is an essential check that models are as they are described. Here we argue that we need to go further and carry out large-scale, systematic and persistent model comparison – where different models of the same phenomena are com- pared against standardised data sets and each other. Lessons for this programme can be gained from the Model Intercomparison Projects (MIP) in the Climate Community and elsewhere. The benefits, lessons and particular difficulties of implementing a similar project in social simulation are discussed, before sketch- ing what such a project might look like. It is time we got our act together!
Keywords: model inter-comparison, policy modelling, replication, reproduction, IPCC, ABM, verification, COVID-19