The purpose of this lecture is to address that conflict, to assess the validity of the arguments of each side and to draw some conclusions and recommendations for the future of social research. Much of the discussion will focus on management research in particular because of its close relationship to practice and its problem and policy oriented concerns.
The third way indicated in the title is not just the ageing buzz word. It is an important approach recognising that social structures and institutions are products of individual behaviour and action. In order to live and work together, people develop norms of behaviour, common beliefs, notions of responsibility, right and duty. While this has been recognised both in the political arena and in the academic literature, social scientists have not yet developed the concept of the third way as a useful approach to policy analysis. In addressing the conflict between economics-style and qualitative social research, a means of developing the third way into a set of practical tools of policy analysis will be defined.
The lecture is organised around the following four points:
Economic analysis is frequently neither expressive nor rigorous. But when it is rigorous, it is surely not expressive. This proposition is demonstrated by means of three examples.
Note that seven points are above and to the left of the main body of points forming a slightly curved line. This set of points stumped Solow until Warren Hogan (1958) pointed out that they were due to an arithmetical error in calculating those data points. Hogan looked for the arithmetical error because he knew what is now common knowledge to those who care to look at the literature: the closeness of the relationship between output and capital, as calculated by Solow’s technique, depends only on the extent to which the distribution of income (i.e., the fractions of income going to wages and profits, respectively) is constant over time. During the period for which Solow took his data, the share of wages in income in the USA was virtually constant at about 65 per cent.
This point was argued verbally by Hogan, who concluded:

So was that the end of Solow’s analysis of technical change and the aggregate production function? Not a bit of it. Since 1981 – the start of the Social Sciences Citation Index data – there have been 454 articles citing Solow’s 1957 article, 319 of them since 1990. During the same period, Hogan’s paper was cited twice and Shaikh’s eight times. All of the citations to Shaikh’s papers were by economists I recognised as heterodox (Sam Bowles and Geoff Hodgson, for example) or there they were in fringe journals (Review of Radical Political Economy, Cambridge Review of Economics). It is safe to say that the critique, accepted in substance by Solow and never successfully refuted, has not prevented the paper and the technique from remaining hugely influential. During the last UK General Election, the present Chancellor of the Exchequer was reported to have said that his policies would be guided by neo-classical endogenous growth theory. That theory is nothing more nor less than an elaboration of the Solow’s technique for measuring technical change.
If there is a theoretical basis for this view, it is general equilibrium theory. In the most general versions of the theory, all transactions are agreed at the start of time. Each transaction is contingent on a set of events pertaining at the time the transaction is to be completed. For example, an individual might contract to buy an umbrella in Duluth, Iowa on January 12, 3944 provided that it is raining and the individual is in Duluth on that day. Since all of these models require there to be a given set of individuals defined by utility functions and probability distributions for the occurrence of every possible event at every date, any individual around at the Creation is assumed to be around forever – including on the date January 12, 3944 (my 2000th birthday).
The first of these models was due to Arrow and Debreu (1954).
The first economist to pursue the consequences of assuming that individuals in such a system could agree and complete transactions at every one of a sequence of dates was Roy Radner (1968). The implication found by Radner for the existence of a general equilibrium when there are spot markets (i.e., markets in which goods are exchanged for money at the same time as, for example, in a shop) is this:
Agents can have rules to determine their supplies and demands for the various goods and services available to them in different circumstances. In the original general equilibrium model of Arrow and Debreu (1954), such rules were not necessary because all the information that would ever be available was conveyed by prices. But when trading takes place and can be revised over time, then the ways in which other individuals behave will affect the consequences to you of your own behaviour. Moreover, the amount of information about other agents and their rules of behaviour grows without limit. Unless individuals all have sufficient computational capacities to calculate their own best rules of behaviour by identifying the rules used by the other individuals (and the effects of those rules on themselves), there can be no general equilibrium. If, however, computational capacities are limited, then eventually there will be more information available than the individuals can use. Consequently, general equilibrium cannot exist unless individuals have unlimited computational capacities.
This would seem to be an important result since it states formally that the theoretical basis of economists’ views of markets requires buyers and sellers in all markets to have unlimited computational capacities.
Though by no means as influential as the Solow paper discussed above, the Social Sciences Citation Index records 68 citations of the Radner paper since 1981. None of those papers address Radner’s conclusion that unlimited computational capacity is a necessary condition for equilibrium when spot trading takes place over time. Instead, they address a more restricted problem identified by Radner relating to the existence of equilibrium when individuals have different information about the environment but there is no spot trading. In that case, he proves that general equilibrium can exist.
The fundamental idea behind (and justification of) the rational expectations theory is that it would be wrong to assume that econometric modellers were smarter than the individuals who make the decisions that generate the time-series data used to specify and estimate the econometrician’s model. If those individuals are really as smart as the econometricians, and if the econometricians’ model of the economy is correct, then the individuals will also have specified the correct model of the economy and it will be the same as that of the econometricians.
One consequence of this line of reasoning is that, if all agents know the correct (econometricians’) model of the economy, they therefore have the same model. By using this model to form their expectations, they must all have the same expectations and there is nothing analytically to distinguish one agent from another. The point can be put in two ways. Either there is a single representative agent or agents are homogeneous. In either case, individuals have nothing to learn about one another and the basis of the Radner problem vanishes.
All of the main models used for forecasting features of the UK economy incorporate the assumption of rational expectations or its close cousin, consistent expectations. Evidently, we can expect these models accurately to reflect the economy if we are either all alike or if there is only one of us.
Tom Sargent, one of the most influential and prolific contributors to rational expectations research, noted in 1993 that (for reasons that are not relevant here) rational expectations theory actually required individuals to know a great deal more about the economic system than the econometricians. He therefore proposed to substitute for individuals with rational expectations individuals with cognitive capacities represented by an optimising algorithm involving the evolution of successful rules of behaviour. These are called genetic algorithms. He then reported a number of simulation models incorporating such agents that converged to a rational expectations equilibrium. This work has given rise to a minor industry among economists that seems to work in the following way:
In every case that I investigated, the purpose of the simplification
was to recast the problem being considered into a form that could be analysed
using conventional economic modelling techniques. In no case was the
technique modified to make it more suitable for analysing the problem.
When it comes to the analysis of serious issues of social policy such
as public investment in education, neither formal rigour, nor logical coherence,
nor any considerations of realism are criteria. No other conclusion is
possible from the importance in the economics literature of Solow’s analysis
of technical change and the aggregate production function.
When analytical rigour is maintained and the most rigorous of economic
analytical forms is used to extend the foundations of economics in a realistic
direction, then fundamental problems thrown up by such developments are
ignored. How else can we interpret the way in which economic theorists
have ignored the Radner proposition that decision makers must have unlimited
cognitive if markets are to function as economic theory requires?
In summary, when economists are concerned with rigour, they cannot be concerned with relevance and when concerned to be relevant, then rigour is not a criterion. Moreover, assumptions for simplicity and the rejection of anecdotal evidence ensure that realism is not an issue.
A top-down approach to the question of whether technique dominates substance begins with the distinction made by the qualitative research community between positivism and phenomenology. A clear initial statement is offered by Easterby-Smith, Thorpe and Lowe (1991, p. 24):
A recent book on information systems and qualitative research (Lee, Liebenau and DeGross, 1997) is a good example of the way in which technique can come to dominate over substance in methodological discussion even in apparently applied academic work. There are chapters on the main approaches to qualitative research. In one, Michael Myers elaborates three broad approaches to critical ethnography: holist, semiotic and behaviouristic with semiotic ethnography subdividing into "thick description" and "ethnoscience". The holists apparently do not worry about the relationship between their own psyches and the ways in which they report their observations. They do insist on a need for empathy with the culture being studied. The semiotic school does not require empathy. "Rather, the ethnographer has to search out and analyse symbolic forms – words, images, institutions, behaviours – with respect to one another and to the whole that they comprise" (Myers, 1997, p. 280). Add to this an overlay of critical hermeneutics where "the object must be a text, or a text-analogue, which in some way is … in one way or another unclear." (Taylor, 1976, p. 153; quoted by Myers, p. 281) Graft onto this notion the hermeneutic circle by which is meant that the whole of a text gives meaning to the parts which in turn determine the meaning of the whole. Myers goes on to assert that the "idea of the hermeneutic circle can be applied to the organisation as a ‘text-analogue’".
So on this account we treat the organisation as analytically equivalent to a text about which we can say nothing objective or definitive. There is no reality and, so, no reality to capture.
Interestingly, Myers goes on in the same chapter to give an account of a critical ethnographic study of the introduction of an information system for a New Zealand mental hospital. In that account, it is clear that different stakeholders have different concerns, views and even fears about the introduction of particular modules of the information system. The introduction of the system was itself a consequence of political decisions taken by the New Zealand government.
Myers’ account of all this is clear, the relationships among the different stakeholder groups are stated explicitly as are the differences in views within stakeholders groups and the reasons for them. After giving this account, Myers then relates various aspects of his account to the critical ethnographic approach. In whatever way the critical ethnographic perspective influenced either Myers’ approach to the elicitation of the relevant stakeholder views and knowledge or the way in which his report was cast, this reader at least was unable to identify any way in which the account itself would have been less interesting or comprehensible without any mention of critical (or any) ethnography elsewhere in the chapter.
The conclusion to be drawn from this reading of that chapter and other articles and textbooks using the approaches of ethnography, grounded theory, case studies and the like is that they are all capable of providing clear, well constructed accounts of social processes that do not depend in any crucial way on the particular approach taken in the research from which those reports emanated. Moreover, these reports do not seem to be of a different genus than the descriptions of events used to specify empirically oriented social simulation models.
| 4.1.1 | The language of analysis should be expressive enough to describe the observable phenomena to be analysed |
| 4.1.2 | Clarity in the expression of relationships |
| 4.1.3 | Integration within a single conceptual framework of evidence of different types and from a range of sources |
| 4.1.4 | Clear conditions of application of the conceptual framework |
| 4.1.5 | Specification of novel, plausible phenomena (that are then observed) |
This criterion includes, but is not the same as, prediction. It is possible for models to be developed in collaboration with domain experts and, in particular, with stakeholders. The models are used to explore with the stakeholders their own understanding of phenomena (at, perhaps, a coarser grain or higher degree of abstraction than ethnographic analysis would be used for). If some possibility is identified that seems plausible to the stakeholders and that they had not previously considered, then the analysis has changed their understanding. If in some circumstances the novel phenomenon should be observed, then confidence in the conceptual framework used to anticipate than phenomenon will naturally be enhanced.
In short, though successful prediction is not a necessary consequence of the use of conceptual frameworks in the social sciences, it is a desirable phenomenon that helps to distinguish the circumstances in which such a framework is relied on.
Perusal of the qualitative management research literature indicates that there is considerable discussion about the techniques of analysis that are appropriate to the analysis of particular phenomena and problems. That is, there is certainly an element of problem-oriented analysis in qualitative social research in general. That seems a likely reason for the rapid turnover in leading-edge approaches in sociology and, generally, qualitative approaches to social science. It also seems plausible that the lack of conventional criteria or any framework within which to discuss and elaborate approaches to the development of analytical technique accounts for what appears to the casual observer to be a lack of direction in qualitative technical development.
Moreover, while the rejection of formal modelling is understandable in light of the reputation generated by economic modelling, the restriction of analytical approach limits the testing of our analytical approaches and therefore the confidence that the users of our analyses of social processes and institutions can place in our conclusions.
An intuitive view of the relationships among the models
and approaches consisdered so far is set out in Figure 4. The Solow
model was clearly less formal and rigorous than the Arrow-Debreu and the
Radner models. In addition, the target representations of the Arrow-Debreu
and Radner models included individual households and firms and, so, were
at much finer grain than the aggregate production function. It is
also clear that the qualitative management models are less formal than
any of the economic models and deal with individuals so that their grain
of analysis is at least as fine as that of the general equilibrium models.
As indicated in Figure 4, there appear to be a large number of qualitative
social models at similar levels of grain and (lack of) formalism while
the economic models tend to be less formal as they are at coarser grain.
Figure 4: Grain and formalism of qualitative and economic modelling
techniques
The purpose of more coarse grained analysis in economics is to generate “rough and ready” representations for use in statistical/econometric analysis. In social simulation modelling, coarseness of grain is used to reduce the number of entities being considered so that the analyst can keep the number of relationships in view down to a “manageable” number.
It has long been known (Miller, 1956) that people can hold in short-term memory between five and nine "chunks" of information at a time. Such "chunks" can be more or less complex depending on the experience of the person with the information and relationships being considered. For our purposes, these cognitive limits are sufficient to justify coarsening the grain of analysis.
More formal analyses reduce the richness and expressiveness of the descriptions and specifications under consideration. An example of the difference was developed for two models by Moss (in press). One was the VDT (virtual design team) model reported by Jin and Levitt (1996) concerning research and development processes in which cooperative action (the requirements for cooperation being captured by a critical path model) must take place within an organisational framework. The objective of the design process and steps required to achieve the design were largely specified in natural language and the sort of codes used to label nodes in a critical path model. The second model was Moss’ (1998) model of critical incident management at North West Water PLC. In this model, too, actions and events were specified in effectively natural language so that events included pumpFailure, intruder, chlorineLeak, and so on. In order to relate these two models to one another and to a third model (Carley and Svoboda, 1996) already cast in formal terms, Moss (in press) restated the VDT and North West Water models so that the environment and the actions available to agents was represented by lists of digits and the effects of actions and interactions among different aspects of the environment were represented by matrices. Using this framework, Moss showed that the North West Water model is a special case of the VDT model and the Carley-Svoboda model is a special case of the North West Water model. Moreover, applied to the same issues of organisational structure, the North West Water and VDT models yielded the same results but the Carley-Svoboda model yielded different results from either of the other models.
The point of formalising the two semantically richer models was to facilitate comparisons between them at the same grain of analysis as the original models. The success of this exercise suggests that formalising representations of social processes can facilitate the development of suites of models with clear relationships among the models in the suite. The models within any suite of models would be cast at different grains of analysis.
The penguins in Figure 5 are icons for real actors. The computers are icons for software representations of the actors. Such software representations are called agents in the computer science and social simulation literatures. By "agent" is meant an autonomous computer program that perceives aspects of its environment, is able to use those perceptions to determine appropriate courses of action and is able to act by affecting its environment.
The schematic of Figures 4 and 5 has no analytical content. In particular, the degree of formalism is not well defined though we know unambiguously that, for example, first order predicate logic represents a higher degree of formalism than does a critical ethnography. Much the same is true of the grain of analysis though a model representing firms as the atomic agent – the agent with no identifiable components – is clearly at a coarser grain of analysis than is a model of one of those firms in which departments or divisions of that firm are the atomic agents.
The finest grain of model indicated in the figure represents a single actor by a single agent.
The grain of the model becomes coarser when the number of agents is smaller than the number of actors they collectively represent. A yet coarser representation is where collections of actors are represented by a single agent.
At the finest grain, as represented in Figure 5, validation will entail comparing the simulated behaviour of an agent with the actually observed behaviour of individuals. A considerable amount of work in this area has been conducted by computational cognitive scientists. In several cases, whole programming architectures – specialised computer programming languages – have been developed to simulation the behaviour of the subjects of psychological experiments. Classic works in this field are the Soar programming architecture based on Newell (1990) and Anderson’s (1993) ACT-R. Validation at this grain of analysis has taken the form of comparisons of experimental and empirical results with outputs from models implemented in these architectures.
Validation at more coarse grained models has involved either the assessment of model outputs by domain experts or comparisons of model outputs with observation. The validation of the VDT model is a particularly spectacular example of the latter. The pilot VDT model was developed to represent the R&D process leading to the production of a space launch vehicle. The model identified two systems that were most likely to fail as a result of organisational relationships and failures of communication. The prototype vehicle went off course minutes after launch and was destroyed by the range safety officer. The subsequent investigation identified failure of both systems identified by the VDT model as be at risk.
A natural approach to model validation would be to use data from ethnographic studies first in the design of the models and agents and then to compare the model outputs with actual episodes. Agents are always specified in CPM models to yield explanations for their simulated actions. These explanations can be compared with the protocols obtained by grounded theorists and ethnographers.
A better approach is compositional verification in which the behaviour of coarse grained models is required to be consistent with the behaviour of models at finer grains. This is achieved by ensuring that, for example, the behaviour of a whole firm as simulated in a model of competition among firms can be replicated by a finer grained model of the firm when the environmental characteristics generated at coarser grain are imposed on the firm model. Similarly, the results obtained in simulation experiments with the firm model can be imposed on a model of a component of the firm to ensure that the model yields the results observed at coarser grain. This decompositional procedure can be taken down to the finest grain model so that the kind of individual behaviour that contributes to the most highly aggregated outcomes can be identified and validated.
Any failure of either verification or validation should focus the attention of the analyst on elements of the social system that have not been understood or appropriately represented.
Though structuration processes were identified as such some 15 years
ago, the only dynamic analyses taking them into account have been social
simulation models such as those of Carley and
Svoboda (1996), Edmonds (1999, in
press) and Moss (1998, Moss
and Sent (1999), 1999). These models capture
(at various degrees of abstraction) processes in which norms and beliefs,
etc.
develop. Well validated models, specified using qualitative social research
techniques as developed by students of management, capture the micro level
processes. The more macro effects of such structures are captured by the
more coarse grained models. The verification process ensures that the representations
of these structures at more macro levels are consistent with the lower
level processes that create them. Moreover, the effects of these structures
in both enabling and constraining action by individuals are likely to require
analysis with both the more coarse grained and the more fine grained models.
My understanding (such as it is) of qualitative social research owes much to discussions with Mark Stubbs, Jeremy Rose, Ray Hackney and Sarah Moss. Bruce Edmonds, Nigel Gilbert and Mark Stubbs read earlier drafts and commented perceptively and usefully. I am grateful to them all.
References
Arrow, K.J. and G. Debreu (1954), "Existence of an Equilibrium for a Competitive Economy", Econometrica, 22, pp. 265-90.
Carley, K. M. and D. Svoboda (1996), "Modeling Organizational Adaptation as a Simulated Annealing Process," Sociological Methods and Research, 25, pp. 138-168.
Easterby-Smith, Thorpe and Lowe (1991), Management Research: An Introduction (London: Sage Publications).
Edmonds, B. (1999), "Gossip, Sexual Recombination and the El Farol Bar: modelling the emergence of heterogeneity", Journal of Artificial Societies and Social Simulation, 2, <http://www.soc.surrey.ac.uk/JASSS/2/3/2.html>.
Edmonds, B. (in press), "Capturing Social Embeddedness: a Constructivist Approach", Adaptive Behaviour, 7.
Friedman, Milton (1953), "The Economics of Positive Methodology" in Essays on Positive Economics (Chicago: University of Chicago Press).
Giddens, Anthony (1984), the Constitution of Society (Cambridge: Polity Press).
Hogan, Warren P. (1958), "Technical Progress and Production Functions", The Review of Economics and Statistics, 40, pp. 407-411.
Jin, Y. and R. Levitt (1996), "The Virtual Design Team: A computational Model of Project Organizations", Computational and Mathematical Organization Theory, 2, pp. 171-195.
Lee, A.S., J. Liebenau and J.I. DeGross (1997). Information systems and Qualitative Research (London: Chapman & Hall).
Miller, G. A. (1956), "The Magic Number Seven, Plus or Minus Two: some Limits on our Capacity for Processing Information", Psychological Review, 63, pp. 81-97.
Moss, Scott (1998). "Critical Incident Management: An Empirically Derived Computational Model", Journal of Artificial Societies and Social Simulation, 1, http://www.soc.surrey.ac.uk/JASSS/1/4/1.html
Moss, Scott (1999), "A Cognitively Rich Methodology for Modelling Emergent Socioeconomic Phenomena" in Thomas Brenner (ed), Computational techniques for Modelling Learning in Economics (Boston: Kluwer Academic Publishers), pp. 363-386.
Moss, Scott (in press), "Canonical Tasks, Environments and Models for Social Simulation", Computational and Mathematical Organization Theory.
Moss, Scott and Esther-Mirjam Sent (1999), "Boundedly versus Procedurally Rational Expectations" in Andrew Hughes Hallett and Peter McAdam (eds), Analyses in Macro Modelling (Boston: Kluwer Academic Publishers), pp. 115-146.
Myers, M.D. (1997), "Critical Ethnography in Information Systems" in Allen, et al. (1997), pp. 276-300.
Newell, A.(1990), Unified Theories of Cognition, (Cambridge MA: Harvard University Press).
Radner, Roy (1968), "Competitive Equilibrium Under Uncertainty", Econometrica, 36, pp. 31-58.
Sargent, T.J. (1993), Bounded Rationality in Macroeconomics, (Oxford: Clarendon Press).
Shaikh, Anwar (1974), "Laws of Production and Laws of Distribution: The Humbug Production Function", The Review of Economics and Statistics 56, pp. 115-120.
Solow, Robert (1957), "Technical Change and the Aggregate Production Function" The Review of Economics and Statistics 39, pp.312-320.
Taylor, C. (1976), "Hermeneutics and Politics" in Critical Sociology: Selected Readings (Harmondswroth: Penguin Books Ltd).