Accuracy of survey responses

From: Lawrence DeBivort (debivort@umd5.umd.edu)
Date: Wed 10 Mar 2004 - 11:52:35 GMT

  • Next message: Keith Henson: "Re: War Gods?"

    Greetings, everyone,

    It may be that there has been more research in this area than I am aware of. My own research was done a long time ago, and I did not follow the matter further.

    Chris, our team uses on-site PCD (performance centered design) to both elicit user requirements and design self-deployable, zero-footprint software. That approach avoids many of the problems that emerge from user surveys and off-site user interviews.

    Best regards, Lawry

    > -----Original Message-----
    > From: fmb-majordomo@mmu.ac.uk [mailto:fmb-majordomo@mmu.ac.uk]On Behalf
    > Of Chris Taylor
    > Sent: Wed, March 10, 2004 4:24 AM
    > To: memetics@mmu.ac.uk
    > Subject: Re: SUSPECT SPAM (exim-4.30.conf) -
    >
    >
    > Thanks Frankie and Lawry. I must admit I'm surprised to hear that this
    > is not a big area of study, although there is that 'survey survey'
    > problem which is rather thorny.
    >
    > I found the idea of just discounting the answers of those who are
    > suspected to be 'cheating' interesting -- kind of like the 'bad day to
    > predict the weather' principle that comes out of complex short term
    > climate models, cos that's all they can manage given the inherent
    > difficulties.
    >
    > Just to put it in context; I'm predominantly a data modeller and user
    > reqs gatherer these days, and getting people to (a) be completely honest
    > about how they and others work, and (b) stop saying they're in favour of
    > good things, and against bad things, is a bugger :)
    >
    > Thanks again guys -- good stuff.
    >
    > Cheers, Chris.
    >
    > Lawrence DeBivort wrote:
    >
    > > Greetings, all,
    > >
    > > Years ago (ok, decades) I did a study within our psych department on how
    > > accurately people were responding to surveys and test
    > instruments. I results
    > > were astonishingly low, and one consequence of the research was
    > that I moved
    > > away from using such instruments and surveys in my own work.
    > >
    > > I note the trouble that Khalil Shikaki got into recently in
    > carrying public
    > > opinion surveys among Palestinians. My sense is that people tended to
    > > respond accurately, there, but didn't like the publication of
    > the results as
    > > they could be construed to lift pressure on the Israelis to
    > deal fairly with
    > > the Palestinians.
    > >
    > > I don't know of any really good discussions on this subject,
    > and there is of
    > > course a meta-problem in carrying out a survey to assess the
    > accuracy with
    > > which people respond to surveys!
    > >
    > > Best regards,
    > > Lawry
    > >
    > >
    > >
    > >>>Does anyone know of a decent review article that addresses people's
    > >>>behaviour when filling in psychometric tests? (i.e. the way they
    > >>>might try to guess what the right sort of answer might be instead of
    > >>>answering honestly, and perhaps arms race style counter-tactics by
    > >>>various designers such as repeating the question with a different
    > >>>formulation).
    > >
    > >
    > >
    > >>>Cheers, Chris.
    > >
    > >
    > >
    > > Frankie: > I was hoping someone else would help you out with
    > this because I
    > > am
    > >
    > >>trying to remember stuff I learned in class over 10 years ago. But
    > >>IIRC some of this stuff ought to give you a place to start.
    >

    =============================================================== This was distributed via the memetics list associated with the Journal of Memetics - Evolutionary Models of Information Transmission For information about the journal and the list (e.g. unsubscribing) see: http://www.cpm.mmu.ac.uk/jom-emit



    This archive was generated by hypermail 2.1.5 : Wed 10 Mar 2004 - 12:01:26 GMT