What Is Statistical Significance and Why Does It Matter

This post answers the question every student encounters when they start analysing data: what is statistical significance, and does your result actually mean anything? It is written for high school students conducting original quantitative research, whether in biology, psychology, economics, or any data-driven field. After reading, you will know how to interpret a p-value, avoid the most common misreading in student papers, and understand what reviewers look for when they evaluate your results. If your research is ready for peer review, the Princeton Journal of Pre-Collegiate Research publishes original student work across all disciplines.

What is statistical significance?

Statistical significance is a measure of whether an observed result in your data is likely to reflect a real effect rather than random chance. It is expressed as a p-value. A result is conventionally considered statistically significant when the p-value is below 0.05, meaning there is less than a 5% probability that the observed result occurred by chance alone, assuming the null hypothesis is true.

That definition matters more than it might seem. Most students learn that a p-value below 0.05 means their result is "significant." What they miss is what that actually means and, just as importantly, what it does not mean.

The p-value does not tell you the size of an effect. It does not confirm your hypothesis is correct. It does not measure how important your finding is. It answers one narrow question: given the data you collected, how likely is it that you would see a result this extreme if there were actually no effect at all?

The 0.05 threshold is a convention, not a law. It was proposed by statistician Ronald Fisher in the 1920s as a rough rule of thumb. The American Statistical Association issued a formal statement in 2016 (Wasserstein and Lazar, The American Statistician) clarifying that a p-value below 0.05 should not be treated as binary proof of a finding. Many journals in psychology and medicine now require effect sizes and confidence intervals alongside p-values for exactly this reason.

For high school researchers, this means two things. First, you need to understand what your p-value actually says before you write your results section. Second, reviewers will notice if you conflate statistical significance with practical importance. That conflation is one of the most common reasons student papers receive revision requests or rejection. You can read more about what drives those decisions in this guide on what makes a research paper get rejected.

What is the difference between statistical significance and practical significance?

Statistical significance tells you whether an effect is detectable. Practical significance tells you whether that effect is meaningful in the real world. A study with a very large sample size can produce a statistically significant result for an effect so small it has no practical relevance. A study with a small sample might miss a genuinely important effect entirely.

Consider a realistic student example. Suppose you survey 400 students about sleep hours and test scores, and you find a statistically significant correlation (p = 0.03). That result clears the 0.05 threshold. But if the correlation coefficient is r = 0.08, the effect is tiny. Sleeping one more hour is associated with less than a one-point improvement in test scores. The result is statistically significant. It is not practically significant.

Effect size statistics, such as Cohen's d for comparing two means or Pearson's r for correlations, quantify how large an effect actually is. Cohen's d of 0.2 is considered small, 0.5 is medium, and 0.8 is large (Cohen, Statistical Power Analysis for the Behavioral Sciences, 1988). Reporting effect sizes alongside p-values is standard practice in publishable research. It is also what separates a paper that passes peer review from one that does not.

A strong results section in a student paper does three things: reports the test statistic and p-value, reports the effect size, and interprets both in plain language relative to the research question. Reviewers reading work submitted to student journals consistently flag papers that report p-values without effect sizes as methodologically incomplete. Understanding what reviewers actually look for is covered in detail in this post on data vs. evidence: what reviewers look for in student research.

What are the most common mistakes students make when reporting statistical significance?

Most errors in student research papers follow predictable patterns. Knowing them in advance saves significant revision time and increases the credibility of your results section.

The first and most common mistake is treating p = 0.05 as a hard pass/fail threshold. Students write "the result was significant (p = 0.049)" and "the result was not significant (p = 0.051)" as if these two outcomes describe fundamentally different realities. They do not. A p-value of 0.051 does not mean there is no effect. It means the evidence did not clear an arbitrary threshold. Report the exact p-value and let the reader assess it in context.

The second mistake is omitting the null hypothesis. Statistical significance only makes sense relative to a specific null hypothesis: the assumption that there is no effect or no difference between groups. If you do not state your null hypothesis clearly in your methods section, your results section has no foundation. Reviewers will ask for it. State it explicitly before you run any test.

The third mistake is running multiple comparisons without correction. If you test ten different variables for significance and use a 0.05 threshold each time, you would expect at least one false positive by chance alone. This is known as the multiple comparisons problem. The Bonferroni correction (dividing your significance threshold by the number of tests) is the standard fix. Failing to apply it when testing multiple hypotheses is a methodological error that peer reviewers are trained to catch.

The fourth mistake is confusing correlation with causation in the discussion section. A statistically significant correlation between two variables does not establish that one causes the other. Stating that your results "show" or "prove" a causal relationship when you have only observational data is an overreach that undermines an otherwise sound paper. Use language like "associated with" or "correlated with" and acknowledge confounding variables explicitly.

How to apply statistical significance correctly in your research paper, step by step

1. State your null hypothesis before collecting data. Write it as a precise, testable statement: for example, "There is no significant difference in mean reaction time between the two groups." Your significance test is designed to evaluate this specific claim.

2. Choose your statistical test before you see the results. The choice of test (t-test, chi-square, ANOVA, correlation) must follow from your data type and research design, not from which test gives you the result you want. Choosing your test after seeing the data is called p-hacking, and it is a form of research misconduct.

3. Set your significance threshold in your methods section. State whether you are using p < 0.05, p < 0.01, or another threshold, and explain why. If you are running multiple comparisons, state your correction method.

4. Report the exact p-value, not just whether it is above or below your threshold. Write "p = 0.03" not "p < 0.05." Exact values give readers and reviewers more information.

5. Calculate and report an effect size. For two-group comparisons, use Cohen's d. For correlations, report Pearson's r or Spearman's rho. For categorical data, use Cramer's V or phi. Free calculators for all of these are available through statistical software packages including JASP, which is free and designed for students.

6. Interpret both statistics in your discussion. Address whether the effect is both statistically and practically significant. If the effect is small, say so. Honest interpretation of a modest result is more credible than overstating what your data shows.

7. Review your paper against the submission guidelines for your target journal before submitting. Different journals have different reporting standards. The submission and review process at PJPCR is outlined in detail for students preparing their first submission.

PJPCR publishes original quantitative and qualitative research across all academic disciplines. If your data analysis is complete and your paper is ready for peer review, review the submission guidelines at princeton-jpcr.org.

Frequently asked questions about statistical significance

What is a p-value in simple terms?

A p-value is the probability of obtaining a result at least as extreme as the one you observed, assuming the null hypothesis is true. A p-value of 0.03 means there is a 3% chance of seeing your result if there were actually no effect. It does not measure the probability that your hypothesis is correct. It measures how surprising your data would be if there were no real effect at all.

How long does it take to get a research paper with statistical analysis peer reviewed?

Peer review timelines vary by journal. At PJPCR, the standard review and publication timeline is 2 to 3 months from submission to decision. This includes initial screening, reviewer assignment, and the revision cycle if revisions are requested. A fast-track option is available for students who need a quicker turnaround, which brings the timeline down to 2 to 4 weeks. PJPCR is a pay-on-acceptance journal, meaning submission and peer review are free, and a publication fee applies only for accepted papers.

Do I need advanced statistics software to conduct publishable research?

No. Many publishable student papers use standard statistical tests that can be run in free tools. JASP is a free, open-source statistics programme designed for students and produces APA-formatted output including effect sizes. Google Sheets and Microsoft Excel handle t-tests, chi-square tests, and correlation analysis. The quality of your statistical reasoning matters far more than the software you use to run the numbers.

What makes a high school research paper statistically credible to reviewers?

A statistically credible paper states its null hypothesis, justifies its choice of test, reports exact p-values alongside effect sizes, and interprets both honestly in the discussion. Reviewers look for consistency between the research question, the statistical method, and the conclusions drawn. Overclaiming causation from correlational data, or reporting significance without effect size, are the two most common reasons papers receive major revision requests. You can also review what peer review at the high school level involves at what is peer review in high school journals.

What kinds of research does PJPCR publish?

PJPCR publishes original, peer-reviewed research by high school students across STEM, social sciences, humanities, and interdisciplinary fields. Quantitative papers with statistical analysis are welcome, as are qualitative, theoretical, and mixed-methods work. The journal is selective and does not guarantee acceptance. All published work is open-access. You can browse the archive and review the submission guidelines at princeton-jpcr.org.

What to take away from this

Statistical significance is not a stamp of approval on your research. It is one specific piece of evidence, with a precise meaning and clear limitations. A p-value below 0.05 tells you that your result is unlikely to be a product of chance alone. It does not tell you the effect is large, important, or causal. Reporting effect sizes alongside p-values, stating your null hypothesis clearly, and interpreting your results honestly are what separate a methodologically sound paper from one that reads like a class assignment.

These are not advanced skills. They are standard practice in any quantitative field, and they are entirely achievable at the high school level with free tools and careful thinking. If your research is complete and your analysis is sound, submit it to PJPCR at princeton-jpcr.org/submit.