{"id":1385,"date":"2023-06-22T02:20:50","date_gmt":"2023-06-22T02:20:50","guid":{"rendered":"https:\/\/content.one.lumenlearning.com\/introstatstest\/chapter\/pair-wise-comparisons-for-anova-fresh-take\/"},"modified":"2025-05-16T23:02:23","modified_gmt":"2025-05-16T23:02:23","slug":"pair-wise-comparisons-for-anova-fresh-take","status":"publish","type":"chapter","link":"https:\/\/content.one.lumenlearning.com\/introstatstest\/chapter\/pair-wise-comparisons-for-anova-fresh-take\/","title":{"raw":"Pair-wise Comparisons for ANOVA - Fresh Take","rendered":"Pair-wise Comparisons for ANOVA &#8211; Fresh Take"},"content":{"raw":"<section class=\"textbox learningGoals\">\r\n<ul>\r\n\t<li><span data-sheets-value=\"{&quot;1&quot;:2,&quot;2&quot;:&quot;Complete an ANOVA hypothesis test for pair-wise comparisons&quot;}\" data-sheets-userformat=\"{&quot;2&quot;:12801,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0,&quot;15&quot;:&quot;arial&quot;,&quot;16&quot;:10}\">Complete pair-wise comparisons for ANOVA<\/span><\/li>\r\n\t<li><span data-sheets-value=\"{&quot;1&quot;:2,&quot;2&quot;:&quot;Calculate a confidence interval and p-value for pair-wise comparisons and explain what it means&quot;}\" data-sheets-userformat=\"{&quot;2&quot;:13057,&quot;3&quot;:{&quot;1&quot;:0},&quot;11&quot;:3,&quot;12&quot;:0,&quot;15&quot;:&quot;arial&quot;,&quot;16&quot;:10}\">Calculate a confidence interval and p-value for pair-wise comparisons and explain what it means<\/span><\/li>\r\n<\/ul>\r\n<\/section>\r\n<h2>Pair-wise Comparisons for ANOVA<\/h2>\r\n<p>Recall the study done by the National Survey of Student Engagement, which found that, on average, college students spend 17 hours per week preparing and studying for their classes.[footnote]Aaron. (2014, July 24). Which college majors study the most? MyMajors. https:\/\/www.mymajors.com\/blog\/college-majors-study\/[\/footnote][footnote]Survey Instruments. (2013). National Survey of Student Engagement. https:\/\/nsse.indiana.edu\/nsse\/survey-instruments\/index.html[\/footnote]<\/p>\r\n<p>Previously, we conducted a one-way ANOVA to determine if there are any statistically significant differences between the means of the following majors:<\/p>\r\n<ul>\r\n\t<li>Arts and Humanities<\/li>\r\n\t<li>STEM<\/li>\r\n\t<li>Education<\/li>\r\n\t<li>Business<\/li>\r\n<\/ul>\r\n<p>Using a 5% level of significance, we rejected the null hypothesis that the means are equal and accepted the alternative hypothesis that at least two means are different.<\/p>\r\n<p>Question! Which means are different? To answer this question, we need to make multiple <strong>pair-wise comparisons<\/strong>.<\/p>\r\n<p>We could begin by comparing business and education. This is an example of a pair-wise comparison. However, there are more!<\/p>\r\n<section class=\"textbox tryIt\">[ohm2_question hide_question_numbers=1]2225[\/ohm2_question]<\/section>\r\n<section class=\"textbox tryIt\">[ohm2_question hide_question_numbers=1]2226[\/ohm2_question]<\/section>\r\n<p><img class=\"aligncenter wp-image-1410 size-full\" src=\"https:\/\/content-cdn.one.lumenlearning.com\/wp-content\/uploads\/sites\/27\/2023\/06\/22022049\/Picture1-3.png\" alt=\"\" width=\"258\" height=\"713\" \/><\/p>\r\n<p>Notice the following key concepts:<\/p>\r\n<ul>\r\n\t<li>\u201cDid the original test reveal that at least one of the colors is linked to acne?\u201d<br \/>\r\n[reveal-answer q=\"848878\"]Show Answer[\/reveal-answer]<br \/>\r\n[hidden-answer a=\"848878\"]No, [latex]P&gt;0.05[\/latex], so there was really no evidence to look at individual colors anyway.[\/hidden-answer]<\/li>\r\n\t<li>\u201cHow many tests did the stick researchers conduct?\u201d<br \/>\r\n[reveal-answer q=\"325651\"]Show Answer[\/reveal-answer]<br \/>\r\n[hidden-answer a=\"325651\"][latex]20[\/latex][\/hidden-answer]<\/li>\r\n\t<li>\u201cWhat is the probability of committing a type I error (saying a color is significant when it is actually not) in <em>each<\/em> test?\u201d<br \/>\r\n[reveal-answer q=\"536925\"]Show Answer[\/reveal-answer]<br \/>\r\n[hidden-answer a=\"536925\"][latex]0.05[\/latex][\/hidden-answer]<\/li>\r\n\t<li>\u201cHow do we know the significance level?\u201d<br \/>\r\n[reveal-answer q=\"579676\"]Show Answer[\/reveal-answer]<br \/>\r\n[hidden-answer a=\"579676\"]Each frame compares [latex]P[\/latex] to [latex]0.05[\/latex].[\/hidden-answer]<\/li>\r\n\t<li>\u201cIs the conclusion adjusted for multiple comparisons?\u201d<br \/>\r\n[reveal-answer q=\"784401\"]Show Answer[\/reveal-answer]<br \/>\r\n[hidden-answer a=\"784401\"]No, it reports a [latex]95\\%[\/latex] confidence level and a [latex]5\\%[\/latex] change for a type I error.[\/hidden-answer]<\/li>\r\n<\/ul>\r\n<section class=\"textbox recall\">We need a method to maintain an overall level of significance even when several tests are performed. We call this the <strong>family-wise<\/strong>\u00a0<strong>error rate<\/strong>. The family-wise error rate is defined as the probability of rejecting at least one of the true null hypotheses. Suppose we perform [latex]m[\/latex] independent hypothesis tests. The probability of making a type I error (at least one false rejection) is: [latex]1-(1-\\alpha)^m[\/latex].<\/section>","rendered":"<section class=\"textbox learningGoals\">\n<ul>\n<li><span data-sheets-value=\"{&quot;1&quot;:2,&quot;2&quot;:&quot;Complete an ANOVA hypothesis test for pair-wise comparisons&quot;}\" data-sheets-userformat=\"{&quot;2&quot;:12801,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0,&quot;15&quot;:&quot;arial&quot;,&quot;16&quot;:10}\">Complete pair-wise comparisons for ANOVA<\/span><\/li>\n<li><span data-sheets-value=\"{&quot;1&quot;:2,&quot;2&quot;:&quot;Calculate a confidence interval and p-value for pair-wise comparisons and explain what it means&quot;}\" data-sheets-userformat=\"{&quot;2&quot;:13057,&quot;3&quot;:{&quot;1&quot;:0},&quot;11&quot;:3,&quot;12&quot;:0,&quot;15&quot;:&quot;arial&quot;,&quot;16&quot;:10}\">Calculate a confidence interval and p-value for pair-wise comparisons and explain what it means<\/span><\/li>\n<\/ul>\n<\/section>\n<h2>Pair-wise Comparisons for ANOVA<\/h2>\n<p>Recall the study done by the National Survey of Student Engagement, which found that, on average, college students spend 17 hours per week preparing and studying for their classes.<a class=\"footnote\" title=\"Aaron. (2014, July 24). Which college majors study the most? MyMajors. https:\/\/www.mymajors.com\/blog\/college-majors-study\/\" id=\"return-footnote-1385-1\" href=\"#footnote-1385-1\" aria-label=\"Footnote 1\"><sup class=\"footnote\">[1]<\/sup><\/a><a class=\"footnote\" title=\"Survey Instruments. (2013). National Survey of Student Engagement. https:\/\/nsse.indiana.edu\/nsse\/survey-instruments\/index.html\" id=\"return-footnote-1385-2\" href=\"#footnote-1385-2\" aria-label=\"Footnote 2\"><sup class=\"footnote\">[2]<\/sup><\/a><\/p>\n<p>Previously, we conducted a one-way ANOVA to determine if there are any statistically significant differences between the means of the following majors:<\/p>\n<ul>\n<li>Arts and Humanities<\/li>\n<li>STEM<\/li>\n<li>Education<\/li>\n<li>Business<\/li>\n<\/ul>\n<p>Using a 5% level of significance, we rejected the null hypothesis that the means are equal and accepted the alternative hypothesis that at least two means are different.<\/p>\n<p>Question! Which means are different? To answer this question, we need to make multiple <strong>pair-wise comparisons<\/strong>.<\/p>\n<p>We could begin by comparing business and education. This is an example of a pair-wise comparison. However, there are more!<\/p>\n<section class=\"textbox tryIt\"><iframe loading=\"lazy\" id=\"ohm2225\" class=\"resizable\" src=\"https:\/\/ohm.one.lumenlearning.com\/multiembedq.php?id=2225&theme=lumen&iframe_resize_id=ohm2225&source=tnh\" width=\"100%\" height=\"150\"><\/iframe><\/section>\n<section class=\"textbox tryIt\"><iframe loading=\"lazy\" id=\"ohm2226\" class=\"resizable\" src=\"https:\/\/ohm.one.lumenlearning.com\/multiembedq.php?id=2226&theme=lumen&iframe_resize_id=ohm2226&source=tnh\" width=\"100%\" height=\"150\"><\/iframe><\/section>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-1410 size-full\" src=\"https:\/\/content-cdn.one.lumenlearning.com\/wp-content\/uploads\/sites\/27\/2023\/06\/22022049\/Picture1-3.png\" alt=\"\" width=\"258\" height=\"713\" \/><\/p>\n<p>Notice the following key concepts:<\/p>\n<ul>\n<li>\u201cDid the original test reveal that at least one of the colors is linked to acne?\u201d\n<div class=\"qa-wrapper\" style=\"display: block\"><button class=\"show-answer show-answer-button collapsed\" data-target=\"q848878\">Show Answer<\/button><\/p>\n<div id=\"q848878\" class=\"hidden-answer\" style=\"display: none\">No, [latex]P>0.05[\/latex], so there was really no evidence to look at individual colors anyway.<\/div>\n<\/div>\n<\/li>\n<li>\u201cHow many tests did the stick researchers conduct?\u201d\n<div class=\"qa-wrapper\" style=\"display: block\"><button class=\"show-answer show-answer-button collapsed\" data-target=\"q325651\">Show Answer<\/button><\/p>\n<div id=\"q325651\" class=\"hidden-answer\" style=\"display: none\">[latex]20[\/latex]<\/div>\n<\/div>\n<\/li>\n<li>\u201cWhat is the probability of committing a type I error (saying a color is significant when it is actually not) in <em>each<\/em> test?\u201d\n<div class=\"qa-wrapper\" style=\"display: block\"><button class=\"show-answer show-answer-button collapsed\" data-target=\"q536925\">Show Answer<\/button><\/p>\n<div id=\"q536925\" class=\"hidden-answer\" style=\"display: none\">[latex]0.05[\/latex]<\/div>\n<\/div>\n<\/li>\n<li>\u201cHow do we know the significance level?\u201d\n<div class=\"qa-wrapper\" style=\"display: block\"><button class=\"show-answer show-answer-button collapsed\" data-target=\"q579676\">Show Answer<\/button><\/p>\n<div id=\"q579676\" class=\"hidden-answer\" style=\"display: none\">Each frame compares [latex]P[\/latex] to [latex]0.05[\/latex].<\/div>\n<\/div>\n<\/li>\n<li>\u201cIs the conclusion adjusted for multiple comparisons?\u201d\n<div class=\"qa-wrapper\" style=\"display: block\"><button class=\"show-answer show-answer-button collapsed\" data-target=\"q784401\">Show Answer<\/button><\/p>\n<div id=\"q784401\" class=\"hidden-answer\" style=\"display: none\">No, it reports a [latex]95\\%[\/latex] confidence level and a [latex]5\\%[\/latex] change for a type I error.<\/div>\n<\/div>\n<\/li>\n<\/ul>\n<section class=\"textbox recall\">We need a method to maintain an overall level of significance even when several tests are performed. We call this the <strong>family-wise<\/strong>\u00a0<strong>error rate<\/strong>. The family-wise error rate is defined as the probability of rejecting at least one of the true null hypotheses. Suppose we perform [latex]m[\/latex] independent hypothesis tests. The probability of making a type I error (at least one false rejection) is: [latex]1-(1-\\alpha)^m[\/latex].<\/section>\n<hr class=\"before-footnotes clear\" \/><div class=\"footnotes\"><ol><li id=\"footnote-1385-1\">Aaron. (2014, July 24). Which college majors study the most? MyMajors. https:\/\/www.mymajors.com\/blog\/college-majors-study\/ <a href=\"#return-footnote-1385-1\" class=\"return-footnote\" aria-label=\"Return to footnote 1\">&crarr;<\/a><\/li><li id=\"footnote-1385-2\">Survey Instruments. (2013). National Survey of Student Engagement. https:\/\/nsse.indiana.edu\/nsse\/survey-instruments\/index.html <a href=\"#return-footnote-1385-2\" class=\"return-footnote\" aria-label=\"Return to footnote 2\">&crarr;<\/a><\/li><\/ol><\/div>","protected":false},"author":8,"menu_order":33,"template":"","meta":{"_candela_citation":"[]","pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[],"contributor":[],"license":[],"part":1348,"module-header":"fresh_take","content_attributions":[],"internal_book_links":[],"video_content":null,"cc_video_embed_content":{"cc_scripts":"","media_targets":[]},"try_it_collection":null,"_links":{"self":[{"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/pressbooks\/v2\/chapters\/1385"}],"collection":[{"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/wp\/v2\/users\/8"}],"version-history":[{"count":5,"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/pressbooks\/v2\/chapters\/1385\/revisions"}],"predecessor-version":[{"id":6854,"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/pressbooks\/v2\/chapters\/1385\/revisions\/6854"}],"part":[{"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/pressbooks\/v2\/parts\/1348"}],"metadata":[{"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/pressbooks\/v2\/chapters\/1385\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/wp\/v2\/media?parent=1385"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/pressbooks\/v2\/chapter-type?post=1385"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/wp\/v2\/contributor?post=1385"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/content.one.lumenlearning.com\/introstatstest\/wp-json\/wp\/v2\/license?post=1385"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}