Received: by alpheratz.cpm.aca.mmu.ac.uk id KAA29574 (8.6.9/5.3[ref pg@gmsl.co.uk] for cpm.aca.mmu.ac.uk from fmb-majordomo@mmu.ac.uk); Mon, 7 May 2001 10:08:46 +0100 Message-ID: <2D1C159B783DD211808A006008062D3101745E5E@inchna.stir.ac.uk> From: Vincent Campbell <v.p.campbell@stir.ac.uk> To: "'memetics@mmu.ac.uk'" <memetics@mmu.ac.uk> Subject: RE: Selection of scientific theories - metascientific experiment Date: Mon, 7 May 2001 10:05:04 +0100 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="iso-8859-1" Sender: fmb-majordomo@mmu.ac.uk Precedence: bulk Reply-To: memetics@mmu.ac.uk
I came across, purely by chance, a book over the weekend that referred to a
study that had done something very similar to what you're suggesting here,
only looking at the hard sciences. Very helpfully, I only recall it now,
without the book to hand. I'll bring it in to work tomorrow and give you
the reference. It really did sound like something very similar to what you
propose.
BTW, I didn't think all your proposed strategies were unethical, only those
that seemed to involve misrepresentation of intent to journal publishers.
You're quite right though, particularly in psychology, there's lots of
experiments where the subjects don't really know what's being tested, and
the experiments are only productive if carried out that way. The fake
electrocution studies spring to mind.
One of my concerns here, was not so much for the poor journal
editors/reviewers, but in terms of motivations for such an approach. You've
made yourself more clear in this post, so I suspect I jumped a little too
readily, but that too can be a risk of such methods. Of course if you're
not bothered about ruffling a few social scientists feathers then that's
fair enough- and you shouldn't let that put you off a piece of empirical
research.
Vincent
> ----------
> From: Metascience
> Reply To: memetics@mmu.ac.uk
> Sent: Saturday, May 5, 2001 8:25 am
> To: memetics@mmu.ac.uk
> Subject: RE: Selection of scientific theories - metascientific
> experiment
>
> At 03-05-01, Vincent Campbell wrote:
>
> >I think you raise interesting questions here, but I'm not so sure of the
> >ethics of your proposed tests of your hypotheses, particularly trying to
> >trick publishers into either publishing joke articles, or into not
> >publishing "good" articles. What would that really show apart from the
> >gullibility of particular journal staff?
>
> Experimenting with people always involves ethic problems. But sometimes
> it's the only way to gain knowledge. For example, every new medicine is
> tested on humans despite considerable risks, because the gains are higher
> than the costs.
>
> In psychology, new therapeutic methods are sometimes applied without any
> testing or systematic evaluation. This is certainly worse, in terms of
> ethics, because harmful therapies may be applied routinely when they
> haven't been tested. This was the case with memory recovery therapy.
>
> There were ethical problems with Sokal's experiment, but he is defending
> it
> anyway. His fake article has certainly had more impact than any other
> article published in 'Social Text'.
>
> I am not going to repeat Sokal's joke, because it has already been done
> excellently. I want to do some of the other proposed studies.
>
> I don't see any ethical problems in submitting a good article to a journal
> and possibly getting it rejected. Neither do I see any problems in sending
> articles to a number of scientists together with a questionnaire.
>
> >Still, you'd need to have very clear
> >definitions of how you define non-falsifiable and falsifiable research,
> and
> >I think you presume too much in this regard.
>
> I need help from experts in philosophy of science. But an experiment could
> still be made which avoids any articles that are borderline or dubious
> with
> respect to this classification.
>
>
> >Also, arguably -
> > <psychological appeal, politics, ideology, funding, tradition,
> > authority, prestige, and sophisticated terminology>
> >all of this can apply to the hard sciences also. To try and argue that
> >they're ok, and everyone else is problematic, and demonstrably so, seems
> >rather specious in fact.
>
> I am not claiming that the hard sciences are perfect. Probably, no science
> is free of irrelevant influence.
>
> The high pressure to 'publish or perish' has tempted many scientists to
> fake results.
>
> Research in medicine and technology is to an increasing degree being
> sponsored by commercial interests, and there have been cases where the
> sponsors attempt to suppres the publication of inconvenient results. Also,
> much of this research is going on inside industrial companies which
> publish
> very selectively, and mostly in the form of patents.
>
> There is also the problem that negative findings are seldom published.
> Imagine that twenty scientists are doing each their experiment. They all
> make a statistical test on the results, and by mere chance one of them
> finds a result that lies within the 5% interval of confidence. This result
> gets published as significant, while the 19 negative results don't.
>
> Thus, there are many mechanisms that may distort science. I just want to
> start my study where I expect the distortion to be highest.
>
> The purpose of my experiment should be simply to find the selection
> criteria that control the development of a particular research tradition.
>
> >Of course, the fundamentally interesting point here, is that the methods
> you
> >are proposing are inherently social scientific ones (e.g. surveys and
> >interviews), and thus in a way will be subject to the same kinds of
> problems
> >that you want to interrogate.
>
> Surveys and interviews are perfectly valid scientific methods. I don't
> want
> to trash the social sciences, I want to improve them.
>
> =============================================
> M. Schwartz, Ph.D.
> Metascience@agner.org
>
> ===============================================================
> This was distributed via the memetics list associated with the
> Journal of Memetics - Evolutionary Models of Information Transmission
> For information about the journal and the list (e.g. unsubscribing)
> see: http://www.cpm.mmu.ac.uk/jom-emit
>
===============================================================
This was distributed via the memetics list associated with the
Journal of Memetics - Evolutionary Models of Information Transmission
For information about the journal and the list (e.g. unsubscribing)
see: http://www.cpm.mmu.ac.uk/jom-emit
This archive was generated by hypermail 2b29 : Mon May 07 2001 - 10:12:22 BST