Re: Selection of scientific theories - metascientific experiment

Date: Fri May 04 2001 - 22:31:56 BST

  • Next message: "Re: memetics-digest V1 #667"

    Received: by id WAA25293 (8.6.9/5.3[ref] for from; Fri, 4 May 2001 22:29:25 +0100
    From: <>
    Date: Fri, 4 May 2001 16:31:56 -0500
    Content-type: text/plain; charset=US-ASCII
    Content-transfer-encoding: 7BIT
    Subject: Re: Selection of scientific theories - metascientific experiment
    Message-ID: <3AF2D97C.29375.995C69@localhost>
    In-reply-to: <>
    X-mailer: Pegasus Mail for Win32 (v3.12c)
    Precedence: bulk

    On 3 May 2001, at 9:52, Metascience wrote:

    I recommend that you obtain, as a reference, Thomas S. Kuhn's
    most recent (postumously published) work, THE ROAD SINCE
    STRUCTURE, which has much to say on this issue.
    > Selection of scientific theories - Proposal for a metascientific
    > research project.
    > It is generally acknowledged that the progress of scientific knowledge
    > is a selection process. Good theories are preserved, while bad
    > theories are rejected. This fits perfectly into a memetics framework.
    > I want to study the selection process that controls the development of
    > the soft sciences, and I am writing this message to solicit your
    > advise on how to carry out this research, your comments on the
    > methodology I am proposing, and possibly your participation and help.
    > The hypothesis I want to test is as follows:
    > In the hard sciences, theories are tested by means of experiments in a
    > well-defined manner. There is almost universal agreement on the
    > criteria for selecting the best theories: logic, experiment,
    > reproducibility, verification, falsification.
    > In the more soft sciences, however, experiments are difficult to carry
    > out and difficult to interpret. The softer the science, the more
    > difficult it is to make rigorous tests because of the complexity and
    > fuzziness of the phenomena. In sciences like psychology and sociology,
    > researchers may refrain from testing their theories through
    > experiments for practical, economic, or ethic reasons, or because they
    > have not been trained to do so. But, obviously, theories within these
    > sciences are still being selected. The important question is now:
    > Which selection criteria are controlling the development of the soft
    > sciences?
    > Social and psychological phenomena are so complex, that any simple
    > theory about cause and effect will have exceptions. Thus, causal or
    > mechanistic theories within these sciences are very vulnerable to
    > falsification. Opponents of a theory can always find an exception,
    > which the theory can't explain. There are two possibilities for
    > dealing with such a problem: (1) refining the theory so that the
    > exception is accounted for, or: (2) rejecting the theory completely.
    > Now, my claim is that certain scientific communities are choosing
    > option (2) so often, that most falsifiable theories are rejected. The
    > long-term outcome of this selection process is that most of the
    > theories that remain are non-falsifiable, and thus not scientific in
    > Popper's sense.
    > I have met many sociologists who completely reject all mechanistic
    > cause-and-effect theories. What remains in their scientific universe
    > is definitions, interpretations, and holistic theories - nothing
    > falsifiable. Paradoxically, they are still paying lip service to
    > Popper's criterion of falsifiability. (The holistic theories say that
    > every phenomenon has an infinite number of causes. Any claim that a
    > certain observation falsifies the theory can always be rejected by
    > saying that some causal factor has not been accounted for).
    > Other selection criteria that control the development of the soft
    > sciences are: psychological appeal, politics, ideology, funding,
    > tradition, authority, prestige, and sophisticated terminology. Thus, a
    > new theory is most likely to be accepted if it appeals emotionally to
    > the referees, if it supports prevailing political ideologies, if it is
    > easy to obtain funding for more research to support the theory, if it
    > is not too far from existing paradigms, if it is supported by
    > reference to the 'big thinkers' who are regarded as authorities within
    > the research tradition, and if the author has a high position and is
    > good at boosting his prestige by mastering a sophisticated vocabulary.
    > These claims are inflicting a hard blow to many research traditions.
    > In fact, they have so far-reaching consequences for the soft sciences
    > that they have to be tested in a more rigorous way than the research
    > they criticize.
    > Therefore, I want to discuss possible ways to test my claims about
    > selection criteria. I can think of the following methods:
    > 1. Study published articles within the research tradition under
    > scrutiny. The advantage of this method is that it is a natural
    > experiment without interference from the experimenter. The
    > disadvantage is that it doesn't tell which articles have been
    > rejected.
    > 2. Ask journal editors for copies of all articles that have been
    > rejected within a certain time period as well as all referee reports.
    > This may be quite a reliable method if editors will cooperate, but
    > most of the rejected articles will probably turn out to have been
    > rejected for good reasons. Finding the original paradigm-breaking
    > contributions that have been rejected will probably be like finding a
    > needle in a haystack.
    > 3. Interviewing referees and editors. While some referees may admit
    > to ideological bias, few will be able to recognize their own
    > susceptibility to emotional appeal, and probably nobody will admit
    > that they don't support the criterion of falsifiability.
    > 4. Send articles to a number of scientists together with a
    > questionnaire asking how they would judge these articles if they were
    > referees on a journal within their field. The questionnaire could ask
    > the relevant questions to elucidate which criteria have influence on
    > the evaluation of the articles. If the return rate is sufficiently
    > high, this experiment could give sufficient data for a statistical
    > analysis. There is still the bias, though, that the scientists know
    > that they are being monitored.
    > 5. Getting bad articles published. This experiment has already been
    > done excellently by physicist Alan Sokal, who got an article published
    > in a sociology journal with the title: "Transgressing the Boundaries:
    > Toward a transformative hermeneutics of quantum gravity" (see ref.
    > below). This article is pure nonsense and parody, but it supports the
    > ideological agenda of the sociology journal.
    > 6. Getting good articles rejected. This would be the ultimate test
    > for my claim that certain research traditions are rejecting
    > falsifiable theories. But there are big problems with such an
    > experiment: Firstly, it would be very presumptuous to assume that we
    > could write an article so good and original that every journal ought
    > to accept it. Secondly, the article would necessarily deviate
    > significantly from the predominant paradigms, and may be rejected
    > simply for this reason. Those scientists who are deeply involved in an
    > existing paradigm are very unlikely to accept a new paradigm,
    > according to Kuhn. To circumvent this problem, we might make the
    > article resemble an existing paradigm as much as possible, we might
    > present it as interdisciplinary or as a new paradigm, or we may dress
    > it up as an improvement to and revival of an old well-known paradigm
    > that has gone out of fashion. Anyway, we would have to argue with the
    > editor and referees after rejection in order to elicit all arguments
    > for rejecting the article.
    > I already have a proposal for a sociology article which would be
    > suited for experiment 4 or 6. I can't reveal the details here because
    > some journal editor or referee might read this mailing list.
    > Anyway, this is a big research project that I can't do alone. I need
    > your suggestions and help.
    > Best regards
    > Metascience
    > Literature references:
    > Popper, K R: Objective knowledge: an evolutionary approach, Oxford
    > 1972.
    > Kuhn, Thomas S: The structure of scientific revolutions. Univ. of
    > Chicago Press, 1962.
    > Sokal, Alan D: Transgressing the boundaries: Toward a transformative
    > hermeneutics of quantum gravity. Social Text 46/47 vol. 14 no. 1-2
    > 1996 p. 217.
    > =============================================
    > M. Schwartz, Ph.D.
    > ===============================================================
    > This was distributed via the memetics list associated with the
    > Journal of Memetics - Evolutionary Models of Information Transmission
    > For information about the journal and the list (e.g. unsubscribing)
    > see:

    This was distributed via the memetics list associated with the
    Journal of Memetics - Evolutionary Models of Information Transmission
    For information about the journal and the list (e.g. unsubscribing)

    This archive was generated by hypermail 2b29 : Fri May 04 2001 - 22:33:00 BST