other modelling papers
approaching agent based modelling | lake anderson with agents | perspectives in agent based social simulation | mental modelling and model moderation | modelling for firma: an example | supporting social simulation | a multi-agent toolkit | how to implement social policies |

perspectives on agent based social simulation: validation, verfication and the firma project : Scott Moss


contents | foundational ABSS: degrees of abstraction | representational ABSS | description as validation | relationship of validation to declarative/imperative distinction |

foundational and representational agent based social simulation

The distinction between these is that foundational ABSS develops tools, approaches and representations for representational ABSS, while representational ABSS describes what we observe.

i) foundational ABSS: degrees of abstraction

These degrees of abstraction range from:

  • Highly formal: BDI logics used to investigate the meaning of core issues, such as trust, helpfulness, etc.
  • Formal but less abstract: deontic logics used to analyse social rules, such as obligation, duty, right, and legal systems.
  • Foundational but concrete: eg Sugarscape.

    These are general descriptions of classes of observations

ii) representational ABSS

This describes target social systems. It complements qualitative descriptions, and is less expressive than (say) ethnographic studies. The relationships are expressed with less ambiguity.

 

validation and verification

  • Validation: determining whether a simulation model is a "good" representation of the target social system
  • Verification: determine whether the model is consistent and sound with respect to some formalism or other model or theory

validation

This is concerned with relating models to target systems, and requires criteria of goodness of the model. This is relevant to representational social simulation, and raises the question of what is a "good" representation of a social system.

There is a choice to be made between two forms of validation, prediction or description.

i) Prediction as Validation

  • Prediction is often impossible in the social sciences. When prediction is required in practical situations, social structures often arise to provide estimates.

    Examples of this include:

    • the rules of the stock exchange
    • rules of commodity markets
    • conventional competitive practices
    • informal procedures and relations in organizations
    • stable friendships and commercial relations.
    • Except for highly constrained cases (e.g., which side of the road to drive on) prediction is not a viable option or goal for the social sciences.

    An IAM view on prediction is expressed by van Asselt and Rotmans: "The future is inherently uncertain and thus unpredictable. The issues presently associated with global change distinguish themselves from familiar scientific problems in several respects: the phenomena, being novel, complex and variable, are themselves not well understood."

ii) Description as validation

In this case, validation is the demonstration that a model is a "good" representation of its target system. Good means telling the truth and nothing but the truth (though not the whole truth). Furthermore, a good model has a subset of the structure and some though not all of the relationships of its target system

 

description and imperative modelling

Imperative modelling specifies processes. Process is inferred from outcomes, and is hard to observe directly. It would amount to direct observation of each step by each component. It is an inherently and extremely fine grain activity

description and declarative modelling

Declarative modelling specifies behaviour: in conditions X, John does Y. These conditions X need not be exhaustive; sufficiency rather than necessity is important. It is easier to observe whether John systematically does Y in some observable conditions X than to observe everything everyone does during some social process.

 

Relationship of validation to declarative/imperative distinction:

  Imperative Declarative
Foundational

Game theory

Sugarscape

Arthur's el Farol Models

BDI approaches

Edmonds' el Farol models

Representational

vanAsselt Rotmans Weissbuch/Images Doran's EOS models (?)

Deontic logic applications Moss' Critical incidents, transitional economy models
  • Representational verification
    • This models at coarser grains of representation and entail agents whose behaviour emerges at finer grain.
  • Foundational verification
    • Agent behaviour sound and consistent with a chosen formalism (eg BDI or deontic logic). Agent behaviour conforms to experimentally validated theory of cognition

'top-down and bottom-up'
  • The top is defined by the largest target system, such as global, regional, national, catchment
  • The bottom less naturally defined, such as organisation, department, individual, neurones
  • System development will develop different levels interactively; but where is the bottom?

validation determines the bottom

Declarative agent-based modelling describes agents' actions in various conditions of the system. The bottom of the system seems likely to be determined by the fineness of grain at which stakeholders or modelling teams can validate the agents as descriptions of target entities (actors, departments, organisations). Independent validation by stakeholders of simulated behaviour occurs at more aggregated levels

 

 


This website is the sole responsiblity of the Firma Project and does not represent the opinion of the European Community nor is the European community responsible for any use that might be made of the data appearing herein.