Controversies in Meme Theory

Rose, N., 1998; Controversies in Meme Theory.
Journal of Memetics - Evolutionary Models of Information Transmission, 2.
http://cfpm.org/jom-emit/1998/vol2/rose_n.html

Abstract

1 Introduction

Evolution by natural selection has been controversial since Darwin first published `The Origin of Species' in 1859, not least amongst those who hold religious beliefs. By demonstrating that through the accumulation of small inherited changes and differential survival, `nature' could select animals upon the basis of their suitability to their environment, Darwin achieved an explanation of how complexity and apparent design could have arisen in nature without the need for a designer. Perhaps the most controversial aspect of Darwin's theory was that it challenged the view that `Man' was somehow separate from the rest of nature. Human beings were merely another branch of life, stemming from the same evolutionary process and primitive beginnings as all other animals.

Despite the controversy, Darwinian evolutionary theory has succeeded in becoming a central feature of the biological sciences. The theory has undergone some refinement during the time it has been around; not least with the unification in the 1930's with the genetic mechanism of heredity (e.g. Fisher [22]). Essentially, however, it could be said that Darwin's theory is the centre piece of biology, the basis for explaining the origins and history of life on Earth, and perhaps everywhere else in the universe (c.f. Dawkins [17]; Dennett [19]).

The core tenets of evolution by natural selection `are as much open to doubt as the theory that the earth goes round the sun' (Dawkins [15], p1). Advances in sociobiology and evolutionary psychology attempt to use the strength of this theory to generate explanations of human behaviour by seeking possible genetically based evolutionary adaptations (e.g. Barkow, Cosmides, and Tooby [2]). However, it is clear that human beings differ from other animals on Earth by the complexity of their culture. By apparently seeking to explain all human behaviour in terms of biology, sociobiology has found itself accused of what Dennett [20] calls `greedy reductionism'. It is clear that much of human culture requires a more appropriate level of description. Although the level of the gene may not fulfil that requirement, it remains to be determined whether evolutionary theory could be more widely applied to incorporate human cultural change. It has been suggested that cultural complexity may have arisen through mechanisms analogous to those responsible for the formation of organic life, leading to the attempt to create a theory for the evolution of culture.

Memetics takes the Darwinian revolution one step further. Evolutionary theory explains how complexity and apparent design could emerge within nature without the need of a designer. In the case of culture the `designer' could be the `common sense' notions of `free will', `intentionality', or `Self'. In a way memetics appears to attempt to replace the `Self' (c.f. Blackmore [5]; Dennett [19]) 1 with an algorithm of culture in the same way that Darwin and his successors have replaced `God' with an algorithm (c.f. Dennett [20], p 50) 2 of life. We can expect the future of memetics to be a difficult and controversial one.

Universal Darwinism (Plotkin [32]; Dawkins [15]) and its application to culture provides a rich and exciting source of speculation from a wide variety of disciplines. Meme theory, in particular, has proved fertile ground for such theorising. Whilst much of this theory has been good, some of it has become confusing and ambiguous.

This paper will offer a critical review of four areas of meme theory. These involve difficulties for the theory which I suspect are also of concern to many people both within, and outside, memetics. I have views as to how each of these problem areas could be settled. However, the purpose of this paper is to give a short review of these areas in the hope of stimulating something of the wider debate required to find solutions for these problems.

2 Replicators and Phenotypes

The definition of a meme is currently ambiguous. A meme can be found variously described as; a unit of imitation (Dawkins [15]), a unit of information residing in a brain (Dawkins [16]), culturally transmitted instructions (Dennett [19, 20]), any permanent pattern of matter or information produced by an act of human intentionality (Csikszentmihalyi [14]), roughly equivalent to ideas or representations (Plotting [32]), a unit of information in a mind whose existence influences events such that copies of itself get created in other minds (Brodie [9]), actively contagious ideas (Lynch [28]), a mental representation (Gabora [23]), a self-replicating element of culture passed on by imitation (Oxford English Dictionary), etc. Without some kind of firm definition the word `meme' becomes almost meaningless (c.f. Wilkins [35]) applied to instructions in brains, information, behaviour, words, mental states, books and all kinds of cultural artefacts without consistency.

Dawkins [15] used genes as an analogy for memes and recognised that the analogy had natural limitations. It is difficult to recognise anything like a memetic allele (although Durham [21], refers to `allomemes') or memetic chromosomes, for instance. Dawkins [15] suggested that the meme is still "drifting clumsily about in its primeval soup" (p 206) much like early replicating molecules, and might have a very different mechanism for competition such as memory space in the brain. Thus Dawkins [15] appears to resist any definition of a vehicle of culture or memetic phenotype.

Wilkins [35] attempts to clarify this analogy by tracing Dawkins' conception of memes from Williams' evolutionary gene and examining Hull's (e.g. Hull [26]) distinction between replicator and interactor. I'd like to take an alternative approach and trace Dawkins' conception of memes from Cloak's [12] work on cultural ethology. Cloak attempted to lay the foundation for applying ethological methods to the study of culture- specific human behaviours. Cloak's [12] definitions are much less confused, and thus perhaps more useful, than the definitions for memes. He calls the set of cultural instructions people carry in their nervous systems, `i-culture'. The material structures, relationships among material structures, and changes in these relationships which are actually brought about or maintained by behaviours of those cultural instructions, he calls, `m-culture'. Features of m-culture thus include features of their behaviour, their technology, their social organisation, and when considered as a set of verbal behaviours, ideologies. Cloak [12] summarised the i-culture-m-culture interaction in a way which makes the link to Dawkins' subsequent formulation of memes and their phenotypes quite apparent.

Dawkins' [15] examples of memes, such as, pots, arches and clothes fashions, would naturally belong to Cloak's [12]) `m-culture'; only the cultural instructions inside the brain which produced these artefacts would be considered the replicator whose `purposes' are being served by these things. Dawkins [16] later clarifies his position by adopting the i-culture-m-culture distinction;

Despite this helpful clarification the distinction between memes and their phenotypes has been overlooked by subsequent authors, adding to the confusion generated by Dawkins' [15] lack of clarity. For an example of such confusion I would like to use Dennett's [19, 20] now famous maxim;

Whilst there is no doubting the shock value upon the reader, Dennett's maxim is quite problematic. One could compare Dennett's [19, 20] statement to the (Batesman [3]) quotation; "a bird is the nest's way of making another nest", in genetics. Dawkins [16] points out that Bateson's remark, which arose from an argument that genetic determinants of development are necessary but not sufficient, is wrong in Dawkins' view because the nest is not a replicator. The gene is the replicator, and both the bird and the nest are phenotypic expressions of those genes. As Dawkins [16] puts it; "A nest, like a bird, is the gene's way of making another gene." (Dawkins [16], p98)

Dennett appears to suggest that we should consider that a library (presumably the `information' housed within a library) is a replicator, and the scholar (a person who studies books) a phenotype. However, Dennett [20] goes on to suggest that language (oral or written) is a `vehicle' for invisible memes, which are the cultural replicators. This would suggest perhaps that Dennett does not hold the position that a library is a replicator, but something like a meme vehicle. In which case we might be better off saying that; libraries, as vehicles for the memes, serve to promote the replication of those memes (among scholars). Not as punchy as the original, but much clearer I think, and also consistent with Cloak's [12] original work, where a library would be a class of m-culture, the function of which is the maintenance and propagation of i-culture.

There are two points I hope to have brought out from this brief review of the problem. Firstly, that some of the problems of defining memes and making a firm distinction between replicator/phenotype, stem from Dawkins' [15] lack of clarity and the overlooking, by subsequent authors, of Cloak's [12] work and Dawkins' [16] correction. Secondly, that Cloak's [12] work is worthy of greater consideration than it currently appears to enjoy, as it generates some rigid definitions and distinctions which much of memetics has so far lacked.

3 Lamarckian Heredity

Why did such usually clear and consistent writers fudge the important issue of replicator/phenotype distinction? I suspect one reason was because Dawkins and contemporaries realised that they were proposing an evolutionary system which appeared to work on Lamarckian-like principles - that is, a system which involved the inheritance of acquired characteristics. Given that Darwinists had spent the best part of 100 years trying to explain why biological evolution was not Lamarckian and trying to undo some of the misconceptions generated by this alternative model, it seems possible that they drew back at the last for fear of resurrecting an old `enemy'.

Lamarckian heredity is now widely rejected as the mechanism for inheritance within biological systems (although there have been occasional scares, see Dawkins, (1982) [17]. Lamarck's theories on evolution and inheritance have been often mis-represented or mis-understood, (c.f. Hull [26]). Within memetics we tend to refer to the fact that culture is acquired by directly copying the phenotype as `Lamarckian' (e.g. Boyd and Richerson [8]). Put simply, the view that a blacksmith's son will inherit the developed arm muscles of the blacksmith, is as simply untrue as the idea that a dog which loses a leg will give birth to three legged puppies. Organisms do not appear to inherit the changes made to the phenotype during the lifetime of the parent organism. Although this is untrue for biological systems, something like Lamarckian inheritance may exist for cultural systems.

Short of some kind of telepathy it is difficult to see how instructions for a cultural behaviour inside one person's brain could be transmitted to another person's brain without the mediation of a phenotype. The mediation of the phenotype may be enough to call cultural inheritance `Lamarckian', but of more interest is the question of whether accidental deviations in the phenotype are acquired by the new host. As Blackmore [4] explains, this is not always the case. She gives the example of telling a story with a hoarse voice. When the new host of the story retells the tale, they do not do so with the same hoarse voice. It may be the case that human beings are very resistant to this kind of `Lamarckian' inheritance, and that changes accidentally acquired by the phenotype are not automatically adopted by potential hosts.

Although some theorists (e.g. Boyd and Richerson [8]; Gabora [23]) appear certain that cultural inheritance is `Lamarckian', others (e.g. Dawkins [16]; Hull [26]; Blackmore [4]) seem to have reservations, and question the relevance of the term when applied to cultural evolution. In one sense there is a causal arrow from phenotype to replicator in that the synthesis of a cultural instruction proceeds from the interaction of a potential host and the phenotype of that instruction; and this may be called `Lamarckian'. In another sense the system appears remarkably resistant to accidental noise within the phenotype, often synthesising the cultural instruction without that acquired characteristic of the phenotype. It is not clear whether cultural inheritance is Lamarckian (in the sense given above), or merely occasionally Lamarckian. Therefore, it seems sensible to conclude that cultural inheritance is mediated through the phenotype, and that the possibility exists that characteristics acquired by the phenotype can also be inherited. It may be better to drop the term `Lamarckian' altogether (c.f. Blackmore [6]).

Does this imply a causal arrow between the phenotype and the replicator? (Dawkins [16] concern). It does, but only in that changes to the phenotype may be synthesised as `mutant' instructions in the brain of a host that comes into contact with it. In this way it appears that the mechanism of transmission is simply another example of the difference between memes and genes. Copying the phenotype (when the phenotype has been accidentally changed) could be said to sometimes cause the creation of mutant memes. This form of `Lamarckism' could be seen as just another potential source of copying infidelity in meme transmission. The `meme-line' for the construction of the original meme phenotype has ended, and a new mutant `meme-line' has sprung up. Whilst this is certainly a `problem' for memes (as all copying infidelity is a problem), it does not necessarily cloud the distinction between memes and their phenotypes or prevent cultural evolution from being considered Darwinian.

Indeed the problem of copying infidelity may give us insight into the deeper mystery of why humans have evolved such complex culture whilst no other animal on Earth has done so. Nature abounds with examples of cultural transmission (e.g. Bonner [7]), but humans appear unique in the complexity and variation of their culture. Many writers have speculated as to the cause of this disparity for example; creativity (e.g. Gabora [23]) or Machiavellian intelligence (e.g. Byrne and Whiten [10]).

Dennett [20] suggests that for cultural evolution to occur the fidelity of transmission must lie within a particular range. The fidelity cannot be 100% because culture would have no variation; which is required for differential survival to drive evolutionary change. The fidelity must still be very high, or else culture would have no continuity between individuals and `Good Tricks' (Dennett [20]) would be lost as quickly as gained, and again evolution would not occur.

One might speculate that at some stage of evolution human imitation progressed to the point where it crossed a fidelity threshold. It became high enough fidelity for `Good Tricks' to survive for some time in the system (c.f. Maynard Smith and Szathmary [29], who model permissible genome size as a function of replication accuracy for viruses). It is easy to forget just how hard imitation is, humans are so good at it. Few other species have been shown to truly imitate one another, and none are any where near as proficient at the task as even human babies (c.f. Heyes [25]; Meltzoff and Moore [30]). If transmission fidelity is a key factor responsible for the complexity of our culture we would expect normal adults to be well within the expected range of fidelity, whereas humans (e.g. autistic children perhaps) and animal groups which do not readily develop complex culture would not.

4 Sociobiology and Memetics

Evolution by natural selection, with the gene fulfilling the role of the replicator, provides formidable explanatory power when applied to biological systems. Dawkins [15, 17] demonstrates how the idea can not only account for complex biological structures, such as the eye, but also social behaviours, such as patterns of aggression. Altruistic behaviour, which appeared a thorn in the side of `nature red in tooth and claw' models of evolution, received special attention. Biologists, (e.g. Hamilton [24]; Trivers [35]; Dawkins [15]; Cronin [13]; Axelrod [1]; Ridley [33]) have used the `gene's eye-view' to provide convincing explanations of altruistic behaviour among animals through mechanisms such as kin selection and reciprocal altruism. Considering evolution from the point of view of the selfish gene has also given new life to explanations of flamboyant features among animals, such as the peacock's tail, by allowing a better understanding of the mechanisms of sexual selection (e.g. Cronin [13]).

Despite the success of sociobiology in illuminating possible evolutionary mechanisms which lead to particular forms of human behaviour, many aspects of culture remain beyond the aegis of neo-Darwinian explanation. It is apparent that the evolution of biological phenotypes cannot completely explain the panoply of behaviours which humans exhibit. Kin selection, sexual selection and reciprocal altruism may well describe the mechanisms which operated to form, for example altruistic behaviour, but much of the specifics of those behaviours are not adequately explained by sociobiology. The mechanisms sociobiologists examine allow us to understand why altruistic behaviour evolved in the first place, but examples like blood-donation, where the donor appears to receive no (biological) benefit in return, or adoption, where parents invest in non-kin, appear to require additional explanations.

There appears to be a feeling, particularly among psychologists and sociologists that perhaps sociobiology has gone too far (e.g. Rose [34]), a resistance to what some see as `greedy reductionism' (Dennett [20]). It is apparent that there needs to be a theory which attempts description of human behaviour at a more appropriate level of analysis.

However, some proponents of meme theory often appear to undervalue sociobiological explanations for behaviour. Meme theory should only be applied where Occam's razor allows. A theory of cultural evolution should incorporate sociobiological findings, only calling upon the addition of cultural mechanisms where they are necessary in order to accurately describe or explain behavioural phenomena. An example from Lynch [28] illustrates this point.

Whether or not this example is true (and there are no sociological references in support of the claim made), Lynch [28] gives us no clue as to why we should ascribe lean-partner preference to purely cultural factors. In the example simply replacing the word `meme' with `gene' could make it a sociobiological argument. Surely genes which promote men to have sexual relationships with younger women (because of their longer reproductive careers) could lead to a `lean-partner preference'.

As well as over-relying upon the power of `vertical' transmission of memes (`vertical' refers to cultural transmission from parents to children, c.f. Cavalli-Sforza and Feldman [11]), this kind of memetics does not tackle the possibility of other mechanisms for culture (e.g. `epigenetic' rules; Lumsden and Wilson [27]), and overlooks possible genetic biases in cultural selection and the explanatory power already available from sociobiology. We should not ignore or attempt to rewrite the advances made by sociobiology. It seems likely that there will be many instances where memes and genes have co-evolved (e.g. Durham [21]) 3, particularly when the majority of cultural transmission has been `vertical'. If we wish to look for examples of apparently `independent' cultural effects on human behaviour, it might be more profitable to look at biologically maladaptive behaviours which have arisen from predominantly `horizontal' transmission (i.e. cultural transmission between non-kin, c.f. Cavalli-Sforza and Feldman [11]). Where cultural practices appear to directly increase the chances of biological reproductive success we should acknowledge them as co-evolutionary systems (e.g. Durham [21]) and recognise the important contribution of biology to many of our behaviours.

5 Self-centered Selectionism

Finally I would like to examine what I believe is a major misconception which has found its way into memetics. I shall call this misconception Self- centered selectionism. Perhaps Dawkins himself provides the most famous example;

On the very last page of The Selfish Gene, Richard Dawkins closes his argument with a call to overthrow the tyranny of the replicators. In doing so it could be claimed that he committed a common, but fundamental, error; he assumed there was `someone' beyond the constructs of the memes and genes who could do the overthrowing.

The idea that a Self, beyond the constructs of the genes and memes, can select or design memes is what I call `Self-centered selectionism'. Self- centered selectionists claim to accept the idea that memes are the unit of cultural heredity, and even that memes have `power' to influence behaviour; but then contradict themselves by claiming that it is the `conscious Self' which selects which memes a person has in `their' brain. This is the sentiment echoed throughout Csikszentmihalyi [14], and Brodie [9]. The `someone' beyond the construct of the memes and genes is `consciousness', they claim. It is apparent that both of them believe that consciousness has the power to select memes in order to fulfil some life goal.

The other idea they both appear to support is akin to the idea of `directed mutation'; a paradoxical position whereby mutation that occurs is not random, but somehow directed towards some goal. The claim that variation is directed by human intentionality appears utterly incompatible with the idea of culture as an essentially Darwinian evolutionary system.

The claim that we can intentionally design and choose memes begs the question; `why do we need an evolutionary theory at all?'. If the processes of meme selection are not `natural', one is perhaps left with `conscious foresight' picking and choosing memes towards some goal. Even if we ignore the very real issue of consciousness doing anything, this is still entirely at odds with the proposal that culture is an evolutionary system.

Given the complexity and variety of life found on Earth it is not surprising, perhaps, that in the past people have looked to that complexity as evidence for the existence of a God or supreme designer. William Paley [31] used the example of finding a watch on a deserted heath, and that from the complexity and apparent design of the watch one could infer the presence of a designer. By analogy, he claimed, because complexity and apparent design exists within nature, nature too must have had a designer. Paley was firm in the claim that; "Arrangement, disposition of parts, subserviency of means to an end, relation of instruments to a use, imply the presence of intelligence and mind", and his belief that; "There cannot be design without a designer". It is this view which has been roundly defeated by the Darwinist revolution within the biological sciences. In biological evolution it is unnecessary to posit some external influence, or guide, or God which directs evolution along. Natural selection can account for the evolution of complexity and design without a designer, without foresight.

If memes are not selected by the same sorts of `natural' processes which affect genes, it becomes difficult to understand how memes could be said to have `phenotypic power', or any other kind of power for that matter. If one believes God meddles in biology, one need not posit evolution to explain the forms we find in biology. Likewise, if one believes that `consciousness' has the foresight and independence to select and direct behaviours towards some goal, one need not posit evolution to explain the forms we find in culture.

People choose memes, but people are constructs of memes and genes. If meme theory is to be given serious consideration then we must first reject the notion that some `central executive Self' can pick and choose among the memes, and refer instead to the sorts of filters (c.f. Dennett [19]) which memes and genes have constructed. Dennett [19] suggests it is the memes which already occupy the brain which influence the brain to accept or reject new memes which come along.

Understanding the `natural' selection pressures upon memes is of fundamental importance to meme theory. To attribute selection to conscious foresight, ignoring that (short of some kind of precognition) foresight does not really exist, not only undermines the value of meme theory as an evolutionary process, it is also a kind of giving up. Meme selection is already a big enough mystery without introducing the complex and enigmatic concept of consciousness. We might as well attribute meme selection to magic, for all the progress that will achieve.

6 Conclusion

I have briefly, and critically, reviewed four important areas which I believe are central to meme theory. The difficulties which I have covered generate concern both within and outside memetics and which need some kind of resolution before major progress can be made towards a science of memetics. Whilst I have aired my views as to how the difficulties in these areas might be settled, these aspects of meme theory can only really be resolved through further debate and eventually research. It is my hope that these short reviews will go some way to stimulating that process.

Notes

1. I mean `Self' in the same sense that Blackmore [5] refers to. Blackmore suggests that there is no `Self' beyond a construct of memes, no one to `have' a belief, or `own' an idea. She suggests that the illusion of a `Self' is itself a meme-complex (i.e. a collection of co-adapted memes) which is `unzipped' by Zen training. This general line of thinking is similar in direction to some of Dennett's ideas in Consciousness Explained [19].

2. An `Algorithm' in the sense that Dennett [20] uses the term is broadly characterised as a `foolproof and somehow "mechanical" procedure' (Dennett [20], p 50). Here I use the term to underline the essential `mindlessness' of the process involved (i.e. not directed by foresight or a `Self').

3. Durham [21] uses the idea of memes in his book `Coevolution'. He provides some excellent examples and analysis where genes and memes have co-evolved in particular cultures. In his view memes can alter the behaviour of humans in such a way that certain genes are favoured over others; e.g. cultural beliefs about dairying and fresh milk consumption affect the reproductive fitness of genes for adult lactose absorption. Durham comes to the conclusion that evolution involves a dual inheritance system, genetic and cultural; which interact with each other in predominantly co- supportive ways. Whilst this may be true when memes and genes share the same vector of transmission (i.e. from parents to kin), it is likely that when memes begin to use lateral transmission (e.g. to non-kin) they can begin to behave more independently of biological advantage, sometimes leading to biologically maladaptive behaviours (c.f. Cloak [12]).

References

[2] Barkow, J.H., Cosmides, L. and Tooby, J. (1992). The Adapted Mind. Evolutionary Psychology and the Generation of Culture. Oxford: Oxford University Press.

[3] Bateson, (1978). Review of the Selfish Gene. Animal Behaviour, 26:316-318.

[4] Blackmore, S (1996). Memes, Minds and Selves. Seminar paper for the LSE lecture series `About Biology'. Nov 28th 1996, (also at: http://www.memes.org.uk/lectures/mms.html)

[5] Blackmore, S (1998). [in press] Waking from the Meme Dream. In: G. Watson., G. Claxton., and S. Batchelor. (Eds), The Psychology of Awakening: Buddhism, Science and Psychotherapy. Prism Press. (also at: http://www.memes.org.uk/meme-lab/DART96.HTM).

[6] Blackmore, S (1999). [in press] The Meme Machine. Oxford: Oxford University Press.

[7] Bonner, J.T. (1980). The Evolution Of Culture In Animals. Princeton, NJ: Princeton University Press.

[8] Boyd, R. and Richerson, P. (1985). Culture and The Evolutionary Process. Chicago: University of Chicago Press.

[10] Byrne, R.W., and Whiten, A. (1988). Machiavellian Intelligence. Oxford: Clarendon Press.

[11] Cavalli-Sforza, L.L. and Feldman, M.W. (1981. Cultural Transmission and Evolution: A Qualitative Approach. Princeton, NJ: Princeton University Press.

[12] Cloak, F.T. (1975). Is a cultural ethology possible? Human Ecology, 3:161-182.

[13] Cronin, H. (1991). The Ant and The Peacock. Cambridge: Cambridge University Press.

[15] Dawkins, R. (1976). The Selfish Gene. Oxford: Oxford University Press. (2nd Edition, 1989).

[16] Dawkins, R. (1982). The Extended Phenotype. Oxford: Oxford University Press.

[18] Dennett, D. (1991). Consciousness Explained. New York: Little, Brown & Co.

[20] Durham, W.H. (1991) Coevolution: Genes, Culture And Human Diversity. Stanford, CA: Stanford University Press.

[21] Fisher, R.A. (1930). The Genetical Theory of Natural Selection. Oxford: Clarendon Press.

[22] Gabora (1997) The origin and evolution of culture and creativity. Journal of Memetics - Evolutionary Models of Information Transmission, 1, http://cfpm.org/jom-emit/1997/vol1/gabora_l.html

[23] Hamilton, W.D. (1964). The genetical theory of social behaviour, Journal of Theoretical Biology, 7:1- 32.

[24] Heyes, C.M. (1993). Imitation, culture and cognition. Animal Behaviour, 46:999 - 1010.

[25] Hull, D.L. (1982). The naked meme. In: H.C.Plotkin (Ed) Learning, Development, and Culture. Chicester: John Wiley & Sons Ltd.

[26] Lumsden, C. And Wilson, E. (1981) Genes, Mind And Culture. Cambridge, MA: Harvard University Press.

[28] Maynard Smith, J. and Szathmary, E. (1995) The Major Transitions in Evolution. Oxford: Oxford University Press.

[29] Meltzoff, A.N. and Moore, M.K. (1977). Imitation of facial and manual gestures by human neonates. Science, 198:75-78.

[30] Paley, William (1812). Natural Theology. [15th Ed. 1815 Printed by S.Hamilton].

[31] Plotkin, H. (1994) Darwin Machines and the Nature of Knowledge. London: Penguin.

[34] Trivers, R.L. (1971).The evolution of reciprocal altruism. Quarterly Review of Biology, 46:35-57.