vanilla85











{February 5, 2012}   Type 1 and Type 2 error, which one is worse?

Type 1 error is when a researcher reports that there is a significant difference when there is not, and the Type 2 error is when a researcher reports there is no significant difference when there actually is one.  It has been said that the type 1 error is worse than the type 2 error (Thomas, 2012) and I am going to discuss whether that is always the case.

An example, a new drug treating cancer is discovered:

  • When committing the type 1 error, the results show the drug is effective when in reality it is not.  The drug has many side effects but because it is effective in treating cancer it is given to patients on a regular basis.  In consequences, people are not treated against cancer but get more health issues which are the cause of an early death.  This shows how serious the type 1 error can be.
  • When committing the type 2 error, the results might show that the drug is not effective in treating cancer when in reality it is.  Therefore the drug is not administered to people and they die because of the cancer.  This shows that committing either of errors might have fatal consequences.

However, consequences of committing any of the errors depend on research.  For example, people who get up early have more energy and are more efficient. The significant difference is found even if there is not any.  In this case, people’s lifestyle is not going to change and energy levels are not affected.  When the type 2 error is committed and the difference is reported even if there is not any, people may start getting early but their energy levels would stay the same. Either way, none of these errors is going to harm anybody.

Researchers try to avoid both types of errors however psychologists tend to be more worried about committing the type 1 error (Howitt and Cramer, 2011) so they reduce the probability of getting the type 1 error but this increases the risk of committing the type 2 error.  Researchers to balance the risks of getting one of those errors use the probability rate of 0.05.  They also increase the sample size what results in decreasing both, type 1 and 2 errors.   According to Lieberman and Cunningham (2009), the real effects should be determined by replication and meta-analysis instead of determining the results from individual studies.  They claim that this method makes Type I errors within individual studies less important because they are self-erasing and fewer Type 2 errors occur.  Neyman and Pearson (1933) argued that Type 2 error should be controlled in scientific research and Ludbrook and Dudley (1998) argued that Type 1 error should be controlled in biomedical research.

This shows that committing either of errors can have serious consequences or no real consequences at all.  Researchers sometimes cannot reduce risk of getting both errors and sometimes they have to decide which one is more serious and would cause more damage. By reducing the risk of committing one of them the risk of committing the other one is increased.  Overall, researchers should be flexible and open-minded while designing a research and doing analysis because which error is worse depends on a research.

Sources:

Statistics without Maths for Psychology, fifth edition.

Introduction to Research Methods in Psychology, third edition.

http://www.creative-wisdom.com/computer/sas/hypothesis.html

http://www.scn.ucla.edu/pdf/Lieberman-Type-II-(2009).pdf

seminar

About these ads


psud63 says:

i think this was an interesting point of view and evaluation of type 1 and type 2 errors, in saying that making these errors isnt always so dangerous and is more flexible than we may think. I do however think that a type 1 error is still worse as it can result in people wasting time and money based on findings that are not actually significant.



psucd3 says:

Type 1 error is far more dangerous than Type 2 error (Scheff, 1963; Gravetter & Wallnau, 2009) due to the fact that Type 1 error can affect people’s health (mental and physical) in research (Scheff, 1963), whereas the worst Type 2 error can do is just make researchers stop following that line of research, they wont stop researching altogether, they’ll just adjust their hypotheses and try again (Gravetter & Forzano, 2009). For example you rightly stated that if we give a patient a drug to cure cancer and it does not actually help them, they be taking a placebo in affect whilst doctors think the drug is working, this is very bad as cancer left untreated can continue to harm the body. Whereas if we say they can’t have a drug because its not working and it actually does, they won’t necessarily die, they will just get another form of treatment. Therefore Type 1 error is much more dangerous as its much better to receive no drug and take other forms of treatment, then take a drug that doesn’t work.

References
Scheff, T. J. (1963). Decision Rules, Types of Error, and Their Consequences in Medical Diagnosis. Behavioral Science, 8(2), 97-107. doi: 10.1002/bs.3830080202
Gravetter, F. J., & Wallnau, L. B. (2009). Statistics for the behavioral sciences (8th ed.). Belmont, CA: Wadsworth
Gravetter, F. J., & Forzano, L. (2009). Research methods for the behavioral sciences (3rd ed.). Belmont, CA: Wadsworth Cengage Learning.



[...] 1)  http://vanilla85.wordpress.com/2012/02/05/type-1-and-type-2-error-which-one-is-worse/#comment-25 [...]



sinesofmadness says:

Despite what has been stated in your post it becomes very clear from your argument that Type 1 error is much more dangerous. The cancer treatment a very profound example with (as you stated) fatal consequences but in nearly all cases I would consider it “worse” to make false information public. Wuensch, (1994) asses the seriousness of Type 2 error and drug treatment also (http://core.ecu.edu/psyc/wuenschk/stathelp/Type-I-II-Errors.htm). Even in the world of science I would still consider Type 1 error worse. When time and money are concerned it is much worse to tell researchers that there is a significant effect when there is not. This publicly-made false information could be the basis of someone’s research. Imagine the effort it took to recruit participants and the thousands of pounds wasted researching an effect that wasn’t significant in the first place.



prpnw says:

I think that Type 1 error is much more important to avoid as I feel that this has the most significant consequences. Saying something works/does something when in actual fact it does not can cause real controversy and confusion. Like you said in the example of cancer…it would be devastating to find out that a so called ‘breakthrough’ was no real breakthrough at all. If you say something works, it has to. You expect this in everday life…a conditioner for you hair apparently makes your hair softer and nourished, you buy it expecting this outcome. You buy a product to “make your house cleaning easier”, you buy it thinking it will. Type 1 error, to me, needs to be avoided like the plague otherwise you are publishing false results and essentially lying to the people who read the publication and believe the findings. Even when the probability is less than 1%, out of 100 experiments at least 1 will contain a false result (Shuttleworth, 2008). I liked an idea that I found on the internet whilst researching type 1 and 2 errors and that is: neither statistical testing or the legal system is perfect and type 1 errors can occur. Juries sometimes make an error and innocent people go to jail, Statisticians would call this a Type 1 error whereas society call it a travesty.



[...] http://vanilla85.wordpress.com/2012/02/05/type-1-and-type-2-error-which-one-is-worse/ The comment said: I think that Type 1 error is much more important to avoid as I feel that this has the most significant consequences. Saying something works/does something when in actual fact it does not can cause real controversy and confusion. Like you said in the example of cancer…it would be devastating to find out that a so called ‘breakthrough’ was no real breakthrough at all. If you say something works, it has to. You expect this in everday life…a conditioner for you hair apparently makes your hair softer and nourished, you buy it expecting this outcome. You buy a product to “make your house cleaning easier”, you buy it thinking it will. Type 1 error, to me, needs to be avoided like the plague otherwise you are publishing false results and essentially lying to the people who read the publication and believe the findings. Even when the probability is less than 1%, out of 100 experiments at least 1 will contain a false result (Shuttleworth, 2008). I liked an idea that I found on the internet whilst researching type 1 and 2 errors and that is: neither statistical testing or the legal system is perfect and type 1 errors can occur. Juries sometimes make an error and innocent people go to jail, Statisticians would call this a Type 1 error whereas society call it a travesty. [...]



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

et cetera
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: