Blog Due 25/03/2012

Are we really measuring what we intend to?

A few weeks ago, I took part in an experiment through SONA and a thought arose that I decided to blog about. This blog will be considering whether psychologists are ever measuring real life behaviours of just perceptions of these behaviours.

The experiment I took part in required me to play a computer game for 20 minutes followed by a few short tasks/questionnaires. The thing I realised while I was completing the computer game was that half way through I became very thirsty and actually thought “If this wasn’t an experiment, I would just pause the game and go get myself a drink”. And then it occured to me that even though I was trying my absolute best in the experiment, I was definitely not giving the experimenter ‘true’ scores as I knew for a fact that I would not have completed the exercise in the same way had I been at home. I also became aware of the fact that I was trying exceptionally hard to do ‘well’ at the game; more so than if I had been playing it by my own accord. This then left me a bit concerned as by doing my best I was making it less likely that the scores would be truly accurate, but at the same time not trying my best just felt wrong.

So something as simple as being a bit thirsty made me wonder if we can ever measure the behaviour someone would do in the ‘real world’. This article provides a lovely explanation as to psychology’s measurement problem. Are we ever really measuring what we are intending to?

When we conduct studies it is difficult to control every confounding variable possible (such as someone being a bit thirsty part way through the experiment) so how can we be certain that we are measuring the intended variable and nothing else.

So perhaps we should just take results from the behaviours that we can see in real life. But taking results from observations also has it’s own huge problems. As well as some observation techniques being arguably subjective, when a person knows that they are being watched do they change their behaviours – whether on purpose or not? But what if we measure behaviours without participants knowing; would covert observations fix this problem? Observations in which participants are unaware of their participation can have huge issues in terms of ethics ( Although the term ‘spying’ may be going a little far, covert observations do pry on the lives of participants somewhat and I personally would not feel comfortable with someone taking notes about any aspect of my life without my prior consent.

A comforting thought is when many participants are involved in a study, the effects of most confounding variables tend to average out, especially if the results of a control group is also analysed.

So even though research methods may have their slight flaws, overall, they do what they need to.


Pictures from:
Rat cartoon:
Control group:


7 thoughts on “Blog Due 25/03/2012

  1. This blog makes a very interesting point, and I feel the same way as I have had similar experiences when doing SONA experiments, for example I took part in a 2 hour study that consisted of sitting in a pitch black room and using the computer which had bright dots flashing on it. After sitting there for so long it began to make me feel very sick and gave me a migraine, this would of obviously effected my score and consequently affected the data of the experiment. And although it is very hard to control counfounding variables, the nature of this particular study seemed to increase them (or at least it felt that way as the study made me feel very ill).

    There is also the factor of social desirability, which may also effect the outcome of a study. For example a person may participate in a study measuring alcohol intake and depression levels, but as a lot of people still hold stigma about mental illness they may not be truthful on their questionnaire or in their interview because they want to appear more socially desirable. However, this is not the case for all experiments. The zimbardo study is a good example of this, as the participants did conform to their roles and it was rather brutal, and to not a lot of people it would not be considered socially desirable.

    I think it is difficult to measure or know for sure that researchers are measuring what they expect to measure or what they intend to measure, all the time. But it is hard to completly diminish things such as confounding factors and social desirability, therefore probably the only way to test the reliability of the results is to repeat the experiments and hope for similar results!

  2. Very good blog, really makes you question whether in most psychology experiments actually are measuring real behaviour, this could have a profound impact on conclusions drawn from studies. For example in Milgram’s obedience study (1963) would participants have stopped shocking the learner if they believed that they weren’t a part of an experiment? Most people (even in them days) would realise that really shocking people in a experiment would be wrong due to ethics therefore may have continued shocking the learner just to play along…however there is much debate about this which I will not go into. I believe that although many psychological studies result in unnatural behaviour being observed, some studies can truly measure what they intend by using certain methodology such as a field experiments. For example, Piliavin, Piliavin and Rodin’s study (1969) observed factors affecting individuals helping behaviour on a train, individuals weren’t aware they were a part of study therefore their behaviour is likely to be an accurate portrayal of their real life behaviour. In your blog you raise the issue of ethics, however if observed in a natural setting such as a train, where people are likely to observe you anyway, the study isn’t that unethical. Therefore I believe if careful, psychologists can ensure they record real behaviours without breaking the ethical guidelines.

    Piliavin et al (1969) –
    Milgram et al (1963) –

  3. Pingback: Blog Comments due 18.04.12 « psychab

  4. I think this issue is a fairly important one for the field of psychology due to the complexity of human behaviour and the means we use to measure it. Coming back to your SONA experiment, it does sound a very tedious method with the inevitable influence of the confounding variables. With your description it is extremely difficult to imagine the usefulness of such computer tasks and whether they really measure what the researcher intends to. The clinical setting reduces the possibility of natural behaviour being measured, thus leading to the measurement of just confounding variables as you afore mentioned.
    The point you have made about repeating experiments to increase reliability is how theories become grounded in psychology. The development of operational definitions from repeated measuring which allow human behaviour to be assessed as different constructs breaks down a particular behaviour into subjective actions, for example some behaviours associated with depression are fatigue, social withdrawal and loss of appetite. However, the development of theories and the operational definitions that link to them often originate from qualitative research as discussed by Eisenhart,%20K.%20Building%20Theories%20from%20Case%20Study%20Research.pdf, but this method is known to be generally subjective and difficult to generalise. Generalisation is also a problem when trying to establish validity, as researchers often aim to apply their findings to a wider population than what they originally tested. The phenomenon of the WEIRD sample means that only a “slice” of the human population is being measured and in order to measure human behaviour more effectively a broader subject pool should be used
    Finally, your point about the observational method being unethical could be reconsidered as the method only breaks ethical guidelines if people are overtly observed without the researcher obtaining their permission beforehand. If the observation is covert and in a real-life setting the research is adhering to ethical guidelines, like you have said though this method is known to be wholly subjective. It is very difficult for researchers to interpret human behaviour because a) the observers may all have different opinions about the behaviour and b) the behaviour could have occurred for so many reasons other than what the researchers are looking for. Generally, I believe that the field of psychology has adapted its’ research methods aiming to establish reliability and validity but the problem comes in the form of humans and their vast individual differences.

  5. I think the way forward for psychology is to conduct more field experiments. As this does ensure that participants are behaving more naturally. Whilst I accept not everyone is happy being observed for research and not knowing it. The BPS does have guidelines on how to conduct research in a public space. They claim it is OK to conduct research in a public space without their knowledge as long as it is taking place in an area where people would expect to be observed. And the ethical guidelines about confidentiality still apply.

    Great blog and awesome cartoons.

  6. Pingback: H/W for TA « columsblog

  7. Pingback: Last blog comments of second year! :D | katepsuc7d

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s