Are we really measuring what we intend to?
A few weeks ago, I took part in an experiment through SONA and a thought arose that I decided to blog about. This blog will be considering whether psychologists are ever measuring real life behaviours of just perceptions of these behaviours.
The experiment I took part in required me to play a computer game for 20 minutes followed by a few short tasks/questionnaires. The thing I realised while I was completing the computer game was that half way through I became very thirsty and actually thought “If this wasn’t an experiment, I would just pause the game and go get myself a drink”. And then it occured to me that even though I was trying my absolute best in the experiment, I was definitely not giving the experimenter ‘true’ scores as I knew for a fact that I would not have completed the exercise in the same way had I been at home. I also became aware of the fact that I was trying exceptionally hard to do ‘well’ at the game; more so than if I had been playing it by my own accord. This then left me a bit concerned as by doing my best I was making it less likely that the scores would be truly accurate, but at the same time not trying my best just felt wrong.
So something as simple as being a bit thirsty made me wonder if we can ever measure the behaviour someone would do in the ‘real world’. This article http://www.pep-web.org/document.php?id=psar.047c.0091a provides a lovely explanation as to psychology’s measurement problem. Are we ever really measuring what we are intending to?
When we conduct studies it is difficult to control every confounding variable possible (such as someone being a bit thirsty part way through the experiment) so how can we be certain that we are measuring the intended variable and nothing else.
So perhaps we should just take results from the behaviours that we can see in real life. But taking results from observations also has it’s own huge problems. As well as some observation techniques being arguably subjective, when a person knows that they are being watched do they change their behaviours http://faculty.valpo.edu/darkkeli/syllabi/methods/p202outlines/ch3.htm – whether on purpose or not? But what if we measure behaviours without participants knowing; would covert observations fix this problem? Observations in which participants are unaware of their participation can have huge issues in terms of ethics (http://www.sociology.org.uk/mpo22f.htm). Although the term ‘spying’ may be going a little far, covert observations do pry on the lives of participants somewhat and I personally would not feel comfortable with someone taking notes about any aspect of my life without my prior consent.
A comforting thought is when many participants are involved in a study, the effects of most confounding variables tend to average out, especially if the results of a control group is also analysed.
So even though research methods may have their slight flaws, overall, they do what they need to.
Rat cartoon: http://www2.smumn.edu/facpages/~dbucknam/
Control group: http://hippieprofessor.wordpress.com/tag/economics/
Gap Between Psychology Theories and their Application?
This week, my blog is going to discuss whether there is a gap between the theories psychology puts forward as scientifically significant and there use in practical life. For example, Cognitive Psychologists have huge amounts of research about how memory works but are these theories put to use in education to help children with their times tables or spelling? Also, are Developmental theories used to ensure that children are given all the possible tools to grow up in the ‘best’ way possible?
There are many theories about why some people often exceed the recommended maximum amount of alcohol they drink. Recently, a study has proposed that two-thirds of a sample of 18-25 year olds could not accurately identify the correct amount of units of alcohol in their regular drinks (http://www.bbc.co.uk/news/uk-england-sussex-17170501). This information could be hugely beneficial as it means that many people may not even realised they may be drinking too much. However, the information is no use if nothing is going to be done about it (such as a campaign about the correct amounts of units).
Clinical theories should be available to everyone in an attempt to prevent some disorders rather than just giving people treatments after they have been diagnosed. According to this report http://www.bbc.co.uk/news/world-europe-jersey-17098556 soldiers are not recieving the help they need until long after they are diagnosed with post traumatic stress disorder. With this disorder it is important to catch it early (http://ptsd.about.com/od/selfhelp/a/Warning_signs.htm) and so with more methods put in place to prevent it rather than treat it, there could be fewer people suffering as much. This later diagnosis could also mean that there are more people out there who are suffering but have not yet been identified so may not receive help in time.
But is the gap really that big? Is the best being done with the resources available?
Psychology is a huge field of research with few limits as every aspect of human behaviour is under inspection in some parts of psychology (http://raymondphilippe.hubpages.com/hub/Psychology-and-its-Importance). With such a huge area of expertise there are bound to be a few cases in which the findings are not immediately, directly applied to real life. However, with the most crucial of findings, changes are seen quickly and often other findings find their place in society after a while.
Dating sites (http://www.bbc.co.uk/news/technology-17017963), at least, seem to be taking psychological theories ‘to heart’…
Psychology, as a science, should benefit everyone, which means we must attempt to find it a place in every day life.
Here are my comments for this week:
Type I or Type II Errors
Error. The last word anyone wants to hear, especially when it comes to the world of science. Science plays such a vital role in the development of society there is little room for mistakes. Therefore, it is very important to know where errors may occur in order to do everything possible to prevent them. This brings me to the most terrifying words to a budding scientist’s ears; Type I and Type II Errors. Now, when we consider the definitions of these errors they sound very similar in terms of their ‘seriousness’, but is one type of error worse than the other?
Type I Errors refer to the times when a person claims that there study/experiment’s results show a significance difference when there is no real significance (http://www.experiment-resources.com/type-I-error.html). Back in the early days of my first year at Bangor University I was under the impression that Type I errors were much worse than Type II. The reason I thought this is because I assumed that on some level it was the researchers ‘fault’ that they claimed a difference when there was not one (or ‘incorrectly rejecting the null hypothesis’). I think that this was because I was very naïve/uninformed/clueless. I thought that a Type I Error meant that a researcher hadn’t done their job as well as they should have (which is almost NEVER the case). When you consider the methods of a scientific study, we use a confidence level of 95% to determine whether we are confident that our results are true. The part that I forgot/didn’t know about is the other 5%. This means that 5% of the time a scientist can complete their experiment following every rule to the letter but they may still make an error when determining the significance of their results. This link http://www.sportsci.org/resource/stats/errors.html provides a lovely way of explaining how (by the definition of the 95% confidence interval) 1 out of 20 times, you will claim a relationship is in the sample when none exists in the population.
Type II Errors on the other hand occur when a claim is made that there isn’t a significant difference when there actually is (the null hypothesis is incorrectly accepted’). This article (http://www.stats.gla.ac.uk/steps/glossary/hypothesis_testing.html#2err) describes how it is difficult to measure the probability of a Type II Error occurring but that it is often due to sample sizes being too small. Consider a pregnancy test, some women may wish to do more than one test for confirmation, a Type II Error would be very unfortunate in this case for some people!
The reason I think that I changed my mind about Type I Errors definitely being the more serious is when I consider real life examples. For example, if we imagine a scientific study testing the effectiveness of a drug for cancer treatment; A Type I Error would lead us to conclude that the drug worked when in fact it did not whereas a Type II Error would claim that the drug didn’t work when it did. I think that in this case both of the errors would be equally as devastating as the other.
However, if we were to consider the trial of a potential criminal, if a Type I Error were to occur, an innocent person would be put in jail but if a Type II Error occurred a guilty person would walk free. In this scenario I find that my opinion has completely switched as to which error may be the most serious. (This article http://intuitor.com/statistics/T1T2Errors.html gives a good account of both types of errors in the Justice System). I believe that if a guilty man were to walk free it would devalue the whole point of the Justice System completely and so is slightly worse than a Type I (although, I do think a Type I Error would be horrific, just not quite as bad as a Type II).
And so the only conclusion that I can make is that both Errors are very bad and can have huge, devastating consequences. Depending on circumstance (and A LOT of personal opinion) one type can be considered more serious than the other but I do not think (anymore!) that one should be avoided more than the other. A delicate balancing act must be attempted in order for results of our scientific results to be as true as possible.
Thank you!! 🙂