{"id":49,"date":"2023-01-19T17:09:46","date_gmt":"2023-01-19T17:09:46","guid":{"rendered":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/chapter\/reading-reporting-experimental-research\/"},"modified":"2025-10-31T17:56:08","modified_gmt":"2025-10-31T17:56:08","slug":"reading-reporting-experimental-research","status":"publish","type":"chapter","link":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/chapter\/reading-reporting-experimental-research\/","title":{"raw":"Statistical Thinking: Learn It 2\u2014Statistical Significance","rendered":"Statistical Thinking: Learn It 2\u2014Statistical Significance"},"content":{"raw":"<h2>Statistical Significance and P-Values<\/h2>\r\n<p>Even when we find patterns in data, often there is still uncertainty in various aspects of the data. For example, there may be potential for measurement errors (even your own body temperature can fluctuate by almost 1 \u00b0F over the course of the day), or we may need to make generalizations about the entire population based on a small snapshot of data. In such cases, we use statistics to help us understand the probability that our data is telling the right story.<\/p>\r\n<section class=\"textbox example\">\r\n<h3><b>Example: Do Babies Prefer Helpers?<\/b><\/h3>\r\n<p class=\"p1\">In a classic study published in <i>Nature<\/i> (Hamlin, Wynn, &amp; Bloom, 2007), researchers asked a fascinating question:<\/p>\r\n<blockquote>Can infants judge others based on helpful or harmful behavior?<\/blockquote>\r\n<p class=\"p1\">In one version of the experiment, 10-month-old babies watched a simple puppet show:<\/p>\r\n<ul>\r\n\t<li class=\"p1\">A wooden \u201cclimber\u201d with googly eyes tried\u2014and failed\u2014to climb a hill.<\/li>\r\n\t<li class=\"p1\">In one scenario, another character <span class=\"s1\">helped<\/span> push the climber up the hill.<\/li>\r\n\t<li class=\"p1\">In another, a different character <span class=\"s1\">pushed the climber back down<\/span>.<\/li>\r\n<\/ul>\r\n<p class=\"p1\">After several repetitions, the infants were shown both toys\u2014the <span class=\"s2\">helper<\/span> and the <span class=\"s2\">hinderer<\/span>\u2014and invited to choose one to play with.[footnote]Hamlin, J. K., Wynn, K., Bloom, P., &amp; Mahajan, N. (2011). How infants and toddlers react to antisocial others. <em>Proceedings of the National Academy of Sciences<\/em>, 108(50), 19931-19936. https:\/\/doi.org\/10.1073\/pnas.1110306108[\/footnote]<\/p>\r\n\r\n[caption id=\"attachment_169\" align=\"aligncenter\" width=\"1024\"]<img class=\"wp-image-169 size-full\" src=\"https:\/\/content-cdn.one.lumenlearning.com\/wp-content\/uploads\/sites\/14\/2023\/01\/01203312\/2.4.2.1.png\" alt=\"Images of the little figures shown to infants in the experiment\u2014a red circle with googly eyes in the main character, and in the first situation, a blue square helps it up a hill, and in the second situation, a yellow triangle pushes it down the hill.\" width=\"1024\" height=\"352\" \/> <strong>Figure 1<\/strong>. In the research study, babies were shown a character trying to climb up a hill. In this case, the red circle was trying to climb the hill and the blue square helped it up, while the yellow triangle pushed it down.[\/caption]\r\n\r\n<p class=\"p1\">Of the <span class=\"s1\">16 infants<\/span> who made a clear choice, <span class=\"s1\">14 picked the helper toy<\/span>.<\/p>\r\n<p class=\"p1\">That seems like a strong preference\u2014but scientists must rule out other explanations before drawing conclusions.<\/p>\r\n<p class=\"p1\">The researchers controlled for:<\/p>\r\n<ul>\r\n\t<li class=\"p1\"><span class=\"s1\">Color and shape:<\/span> Each toy took turns being the helper or hinderer.<\/li>\r\n\t<li class=\"p1\"><span class=\"s1\">Position:<\/span> Half of the infants saw the helper on the left, half on the right.<\/li>\r\n\t<li class=\"p1\"><span class=\"s1\">Familiarity:<\/span> All infants saw the same number of helping and hindering acts.<\/li>\r\n<\/ul>\r\n<p class=\"p1\">After accounting for these factors, one question remained: Could the result still be due to <span class=\"s1\">random chance<\/span>?<\/p>\r\n<\/section>\r\n<h3><b>Understanding Randomness in Research<\/b><\/h3>\r\n<p class=\"p3\">Even if all infants had <span class=\"s2\">no real preference<\/span>, we wouldn\u2019t expect exactly half to pick each toy every time. Random variation happens naturally.<\/p>\r\n<p class=\"p3\">For instance, if each baby\u2019s choice were like <span class=\"s2\">flipping a coin<\/span>\u201450% chance helper, 50% chance hinderer\u2014we might sometimes get 9 \u201chelper\u201d choices out of 16, sometimes 7, sometimes 11. But getting <span class=\"s2\">14 out of 16<\/span> \u201chelper\u201d choices would be very unusual if the choices were purely random.<\/p>\r\n<section class=\"textbox tryIt\">[ohm2_question height=\"350\"]3428[\/ohm2_question]<\/section>\r\n<h3>P-value<\/h3>\r\n<p>Getting 14 (or more) heads in 16 tosses is about as likely as tossing a coin and getting 9 heads in a row. This probability is referred to as a <strong>p-value<\/strong>. The p-value represents the likelihood that experimental results happen by chance.\u00a0<\/p>\r\n<h3><b>What Does \u201cStatistically Significant\u201d Mean?<\/b><\/h3>\r\n<p class=\"p3\">Psychologists typically use a standard of <span class=\"s2\"><b>p &lt; .05<\/b><\/span> (less than a 5% chance that the result happened randomly) to decide whether results are <strong>statistically significant<em>.<\/em><\/strong><\/p>\r\n<p class=\"p3\">If the probability of getting the observed result by chance is smaller than 5%, researchers conclude that the effect is <span class=\"s2\">unlikely to be random<\/span>. Because <span class=\"s2\">p = 0.0021<\/span> is far smaller than .05, the researchers concluded that infants showed a <span class=\"s2\">genuine preference for the helper toy<\/span>. In other words, the babies\u2019 choices likely reflect something real about social evaluation\u2014not luck.<\/p>\r\n<p>Statistical significance doesn\u2019t \u201cprove\u201d a theory\u2014it simply tells us that the pattern we observed is very unlikely to have happened by accident.<br \/>\r\nResearchers still need to replicate studies and consider effect size, design quality, and potential biases before drawing firm conclusions.<\/p>\r\n<section class=\"textbox tryIt\">[ohm2_question height=\"650\"]3429[\/ohm2_question]<\/section>","rendered":"<h2>Statistical Significance and P-Values<\/h2>\n<p>Even when we find patterns in data, often there is still uncertainty in various aspects of the data. For example, there may be potential for measurement errors (even your own body temperature can fluctuate by almost 1 \u00b0F over the course of the day), or we may need to make generalizations about the entire population based on a small snapshot of data. In such cases, we use statistics to help us understand the probability that our data is telling the right story.<\/p>\n<section class=\"textbox example\">\n<h3><b>Example: Do Babies Prefer Helpers?<\/b><\/h3>\n<p class=\"p1\">In a classic study published in <i>Nature<\/i> (Hamlin, Wynn, &amp; Bloom, 2007), researchers asked a fascinating question:<\/p>\n<blockquote><p>Can infants judge others based on helpful or harmful behavior?<\/p><\/blockquote>\n<p class=\"p1\">In one version of the experiment, 10-month-old babies watched a simple puppet show:<\/p>\n<ul>\n<li class=\"p1\">A wooden \u201cclimber\u201d with googly eyes tried\u2014and failed\u2014to climb a hill.<\/li>\n<li class=\"p1\">In one scenario, another character <span class=\"s1\">helped<\/span> push the climber up the hill.<\/li>\n<li class=\"p1\">In another, a different character <span class=\"s1\">pushed the climber back down<\/span>.<\/li>\n<\/ul>\n<p class=\"p1\">After several repetitions, the infants were shown both toys\u2014the <span class=\"s2\">helper<\/span> and the <span class=\"s2\">hinderer<\/span>\u2014and invited to choose one to play with.<a class=\"footnote\" title=\"Hamlin, J. K., Wynn, K., Bloom, P., &amp; Mahajan, N. (2011). How infants and toddlers react to antisocial others. Proceedings of the National Academy of Sciences, 108(50), 19931-19936. https:\/\/doi.org\/10.1073\/pnas.1110306108\" id=\"return-footnote-49-1\" href=\"#footnote-49-1\" aria-label=\"Footnote 1\"><sup class=\"footnote\">[1]<\/sup><\/a><\/p>\n<figure id=\"attachment_169\" aria-describedby=\"caption-attachment-169\" style=\"width: 1024px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-169 size-full\" src=\"https:\/\/content-cdn.one.lumenlearning.com\/wp-content\/uploads\/sites\/14\/2023\/01\/01203312\/2.4.2.1.png\" alt=\"Images of the little figures shown to infants in the experiment\u2014a red circle with googly eyes in the main character, and in the first situation, a blue square helps it up a hill, and in the second situation, a yellow triangle pushes it down the hill.\" width=\"1024\" height=\"352\" srcset=\"https:\/\/content-cdn.one.lumenlearning.com\/wp-content\/uploads\/sites\/14\/2023\/01\/01203312\/2.4.2.1.png 1024w, https:\/\/content-cdn.one.lumenlearning.com\/wp-content\/uploads\/sites\/14\/2023\/01\/01203312\/2.4.2.1-300x103.png 300w, https:\/\/content-cdn.one.lumenlearning.com\/wp-content\/uploads\/sites\/14\/2023\/01\/01203312\/2.4.2.1-768x264.png 768w, https:\/\/content-cdn.one.lumenlearning.com\/wp-content\/uploads\/sites\/14\/2023\/01\/01203312\/2.4.2.1-65x22.png 65w, https:\/\/content-cdn.one.lumenlearning.com\/wp-content\/uploads\/sites\/14\/2023\/01\/01203312\/2.4.2.1-225x77.png 225w, https:\/\/content-cdn.one.lumenlearning.com\/wp-content\/uploads\/sites\/14\/2023\/01\/01203312\/2.4.2.1-350x120.png 350w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption id=\"caption-attachment-169\" class=\"wp-caption-text\"><strong>Figure 1<\/strong>. In the research study, babies were shown a character trying to climb up a hill. In this case, the red circle was trying to climb the hill and the blue square helped it up, while the yellow triangle pushed it down.<\/figcaption><\/figure>\n<p class=\"p1\">Of the <span class=\"s1\">16 infants<\/span> who made a clear choice, <span class=\"s1\">14 picked the helper toy<\/span>.<\/p>\n<p class=\"p1\">That seems like a strong preference\u2014but scientists must rule out other explanations before drawing conclusions.<\/p>\n<p class=\"p1\">The researchers controlled for:<\/p>\n<ul>\n<li class=\"p1\"><span class=\"s1\">Color and shape:<\/span> Each toy took turns being the helper or hinderer.<\/li>\n<li class=\"p1\"><span class=\"s1\">Position:<\/span> Half of the infants saw the helper on the left, half on the right.<\/li>\n<li class=\"p1\"><span class=\"s1\">Familiarity:<\/span> All infants saw the same number of helping and hindering acts.<\/li>\n<\/ul>\n<p class=\"p1\">After accounting for these factors, one question remained: Could the result still be due to <span class=\"s1\">random chance<\/span>?<\/p>\n<\/section>\n<h3><b>Understanding Randomness in Research<\/b><\/h3>\n<p class=\"p3\">Even if all infants had <span class=\"s2\">no real preference<\/span>, we wouldn\u2019t expect exactly half to pick each toy every time. Random variation happens naturally.<\/p>\n<p class=\"p3\">For instance, if each baby\u2019s choice were like <span class=\"s2\">flipping a coin<\/span>\u201450% chance helper, 50% chance hinderer\u2014we might sometimes get 9 \u201chelper\u201d choices out of 16, sometimes 7, sometimes 11. But getting <span class=\"s2\">14 out of 16<\/span> \u201chelper\u201d choices would be very unusual if the choices were purely random.<\/p>\n<section class=\"textbox tryIt\"><iframe loading=\"lazy\" id=\"ohm3428\" class=\"resizable\" src=\"https:\/\/ohm.one.lumenlearning.com\/multiembedq.php?id=3428&theme=lumen&iframe_resize_id=ohm3428&source=tnh&show_question_numbers\" width=\"100%\" height=\"350\"><\/iframe><\/section>\n<h3>P-value<\/h3>\n<p>Getting 14 (or more) heads in 16 tosses is about as likely as tossing a coin and getting 9 heads in a row. This probability is referred to as a <strong>p-value<\/strong>. The p-value represents the likelihood that experimental results happen by chance.\u00a0<\/p>\n<h3><b>What Does \u201cStatistically Significant\u201d Mean?<\/b><\/h3>\n<p class=\"p3\">Psychologists typically use a standard of <span class=\"s2\"><b>p &lt; .05<\/b><\/span> (less than a 5% chance that the result happened randomly) to decide whether results are <strong>statistically significant<em>.<\/em><\/strong><\/p>\n<p class=\"p3\">If the probability of getting the observed result by chance is smaller than 5%, researchers conclude that the effect is <span class=\"s2\">unlikely to be random<\/span>. Because <span class=\"s2\">p = 0.0021<\/span> is far smaller than .05, the researchers concluded that infants showed a <span class=\"s2\">genuine preference for the helper toy<\/span>. In other words, the babies\u2019 choices likely reflect something real about social evaluation\u2014not luck.<\/p>\n<p>Statistical significance doesn\u2019t \u201cprove\u201d a theory\u2014it simply tells us that the pattern we observed is very unlikely to have happened by accident.<br \/>\nResearchers still need to replicate studies and consider effect size, design quality, and potential biases before drawing firm conclusions.<\/p>\n<section class=\"textbox tryIt\"><iframe loading=\"lazy\" id=\"ohm3429\" class=\"resizable\" src=\"https:\/\/ohm.one.lumenlearning.com\/multiembedq.php?id=3429&theme=lumen&iframe_resize_id=ohm3429&source=tnh&show_question_numbers\" width=\"100%\" height=\"650\"><\/iframe><\/section>\n<hr class=\"before-footnotes clear\" \/><div class=\"footnotes\"><ol><li id=\"footnote-49-1\">Hamlin, J. K., Wynn, K., Bloom, P., &amp; Mahajan, N. (2011). How infants and toddlers react to antisocial others. <em>Proceedings of the National Academy of Sciences<\/em>, 108(50), 19931-19936. https:\/\/doi.org\/10.1073\/pnas.1110306108 <a href=\"#return-footnote-49-1\" class=\"return-footnote\" aria-label=\"Return to footnote 1\">&crarr;<\/a><\/li><\/ol><\/div>","protected":false},"author":20,"menu_order":25,"template":"","meta":{"_candela_citation":"[{\"type\":\"cc\",\"description\":\"Analyzing Findings\",\"author\":\"OpenStax College\",\"organization\":\"\",\"url\":\"https:\/\/openstax.org\/books\/psychology-2e\/pages\/2-3-analyzing-findings\",\"project\":\"\",\"license\":\"cc-by\",\"license_terms\":\"Download for free at https:\/\/openstax.org\/books\/psychology-2e\/pages\/1-introduction.\"},{\"type\":\"cc\",\"description\":\"Statistical Thinking\",\"author\":\"Beth Chance and Allan Rossman \",\"organization\":\"California Polytechnic State University, San Luis Obispo\",\"url\":\"http:\/\/nobaproject.com\/modules\/statistical-thinking\",\"project\":\"The Noba Project\",\"license\":\"cc-by-nc-sa\",\"license_terms\":\"http:\/\/nobaproject.com\/license-agreement\"},{\"type\":\"cc\",\"description\":\"Histogram\",\"author\":\"Fisher\\'s Iris flower data set\",\"organization\":\"Wikipedia\",\"url\":\"https:\/\/en.wikipedia.org\/wiki\/Wikipedia:Meetup\/DC\/Statistics_Edit-a-thon#\/media\/File:Fisher_iris_versicolor_sepalwidth.svg\",\"project\":\"\",\"license\":\"cc-by-sa\",\"license_terms\":\"\"}]","pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[],"contributor":[],"license":[],"part":22,"module-header":"learn_it","content_attributions":null,"internal_book_links":[],"video_content":null,"cc_video_embed_content":{"cc_scripts":"","media_targets":[]},"try_it_collection":null,"_links":{"self":[{"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/pressbooks\/v2\/chapters\/49"}],"collection":[{"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/wp\/v2\/users\/20"}],"version-history":[{"count":21,"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/pressbooks\/v2\/chapters\/49\/revisions"}],"predecessor-version":[{"id":7055,"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/pressbooks\/v2\/chapters\/49\/revisions\/7055"}],"part":[{"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/pressbooks\/v2\/parts\/22"}],"metadata":[{"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/pressbooks\/v2\/chapters\/49\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/wp\/v2\/media?parent=49"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/pressbooks\/v2\/chapter-type?post=49"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/wp\/v2\/contributor?post=49"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/content.one.lumenlearning.com\/introductiontopsychology\/wp-json\/wp\/v2\/license?post=49"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}