Are we really measuring what we intend to?
A few weeks ago, I took part in an experiment through SONA and a thought arose that I decided to blog about. This blog will be considering whether psychologists are ever measuring real life behaviours of just perceptions of these behaviours.
The experiment I took part in required me to play a computer game for 20 minutes followed by a few short tasks/questionnaires. The thing I realised while I was completing the computer game was that half way through I became very thirsty and actually thought “If this wasn’t an experiment, I would just pause the game and go get myself a drink”. And then it occured to me that even though I was trying my absolute best in the experiment, I was definitely not giving the experimenter ‘true’ scores as I knew for a fact that I would not have completed the exercise in the same way had I been at home. I also became aware of the fact that I was trying exceptionally hard to do ‘well’ at the game; more so than if I had been playing it by my own accord. This then left me a bit concerned as by doing my best I was making it less likely that the scores would be truly accurate, but at the same time not trying my best just felt wrong.
So something as simple as being a bit thirsty made me wonder if we can ever measure the behaviour someone would do in the ‘real world’. This article http://www.pep-web.org/document.php?id=psar.047c.0091a provides a lovely explanation as to psychology’s measurement problem. Are we ever really measuring what we are intending to?
When we conduct studies it is difficult to control every confounding variable possible (such as someone being a bit thirsty part way through the experiment) so how can we be certain that we are measuring the intended variable and nothing else.
So perhaps we should just take results from the behaviours that we can see in real life. But taking results from observations also has it’s own huge problems. As well as some observation techniques being arguably subjective, when a person knows that they are being watched do they change their behaviours http://faculty.valpo.edu/darkkeli/syllabi/methods/p202outlines/ch3.htm – whether on purpose or not? But what if we measure behaviours without participants knowing; would covert observations fix this problem? Observations in which participants are unaware of their participation can have huge issues in terms of ethics (http://www.sociology.org.uk/mpo22f.htm). Although the term ‘spying’ may be going a little far, covert observations do pry on the lives of participants somewhat and I personally would not feel comfortable with someone taking notes about any aspect of my life without my prior consent.
A comforting thought is when many participants are involved in a study, the effects of most confounding variables tend to average out, especially if the results of a control group is also analysed.
So even though research methods may have their slight flaws, overall, they do what they need to.
Rat cartoon: http://www2.smumn.edu/facpages/~dbucknam/
Control group: http://hippieprofessor.wordpress.com/tag/economics/