What is effect size and why reviewers care

TL;DR

• Effect size measures how meaningful a result is, not just whether it exists.

• Statistical significance alone does not satisfy peer reviewers anymore.

• Small effect sizes can still matter if the sample is large enough.

• Reporting effect size correctly separates strong papers from weak ones.

• Common effect size measures include Cohen's d, r, and eta-squared.

You ran your study. You got a p-value below 0.05. You feel confident. Then a reviewer sends back a comment asking why you did not report effect size, and suddenly the result you were proud of feels incomplete. This is one of the most common surprises for student researchers encountering serious peer review for the first time.

The confusion is understandable. Most introductory statistics courses teach hypothesis testing and significance thresholds. Effect size often gets a brief mention, if it gets mentioned at all. But journals that publish rigorous empirical research treat effect size as a core part of any result, not an optional add-on.

Understanding what effect size is, how to calculate it, and how to write about it clearly will make your paper stronger at every stage of review. That is what this post covers.

What Is Effect Size and Why Reviewers Care About It

Effect size is a numerical measure of the practical magnitude of a result. It tells you how large, strong, or meaningful a difference or relationship is in your data. A statistically significant result tells you that an effect probably exists. Effect size tells you how big that effect actually is. Reviewers care because a tiny effect can reach significance with a large enough sample, making it statistically real but practically meaningless.

To understand why this matters, consider a simple example. Suppose you study whether listening to classical music before an exam improves test scores. You test 2,000 students and find a statistically significant result. But when you calculate the effect size, you find that the music group scored only 0.3 points higher on a 100-point test. That difference is real. It is also essentially useless for any practical purpose. Without the effect size, a reader might assume the intervention was meaningful. With it, they can judge for themselves.

This is why the American Psychological Association (APA), in its Publication Manual (7th edition), states that authors should always provide effect sizes when reporting the results of hypothesis tests. The APA has included this guidance since the 5th edition, published in 2001. Many journals in psychology, education, medicine, and the social sciences now require effect size reporting as a condition of publication.

If you are working toward submitting your first paper and want structured support navigating reviewer expectations like this one, joining the Publication Compass waitlist puts you first in line for a platform built specifically to help student researchers prepare submission-ready work.

How Effect Size Differs From Statistical Significance

Statistical significance and effect size measure two completely different things. Significance answers the question: could this result have occurred by chance? Effect size answers the question: how large is this result in the real world? You need both to make a complete claim. Reporting only one of them leaves a gap that experienced reviewers will flag immediately.

A p-value is shaped by sample size. As your sample grows, even the smallest true difference will eventually produce a significant p-value. This is not a flaw in the test. It is simply how probability works. But it means that in large studies, significance is almost guaranteed for any effect that exists at all. Effect size is not inflated by sample size in the same way. A Cohen's d of 0.2 means roughly the same thing whether you tested 50 people or 5,000.

Reviewers who work on journals with rigorous methodological standards, such as Psychological Science or PLOS ONE, have seen thousands of papers that report significant p-values for trivial effects. They have learned to look past the asterisk and go straight to the effect size. If it is not there, that is a problem. If it is there but small, they want to see you acknowledge it and explain why the result still matters.

Understanding the full peer review process helps here. If you want a clear picture of what happens to your paper once it is submitted, what is peer review and what happens to your paper walks through each stage in plain terms.

The Most Common Effect Size Measures You Need to Know

The right effect size measure depends on your study design. There is no single universal formula. Choosing the wrong one is itself a methodological error that reviewers will catch. Here are the three measures that appear most often in student research across the social sciences, psychology, and education.

1. Cohen's d is used when comparing two group means, such as a control group versus an experimental group. It expresses the difference between the means in units of standard deviation. Jacob Cohen, who developed this measure, proposed benchmarks in his 1988 book Statistical Power Analysis for the Behavioral Sciences: 0.2 is small, 0.5 is medium, and 0.8 is large. These benchmarks are rough guides, not rules. Context matters more than the label.

2. Pearson's r is used when you are measuring the relationship between two continuous variables. It ranges from -1 to 1. An r of 0.1 is considered small, 0.3 is medium, and 0.5 is large, again following Cohen's conventions. If your study involves correlation, r is usually the appropriate choice.

3. Eta-squared (eta squared) is used in analysis of variance (ANOVA) designs. It represents the proportion of total variance in the outcome that is explained by the grouping variable. Values of 0.01, 0.06, and 0.14 correspond to small, medium, and large effects respectively. Partial eta-squared is a related measure often reported in factorial designs.

There are other measures, including omega-squared, Hedges' g, and Cramer's V, each suited to specific designs. Your choice should follow the conventions of your field. Reading published papers in your target journal is the fastest way to learn what that journal expects. If you are submitting to a journal like the Journal of Research in Science Teaching or Frontiers for Young Minds, look at three or four recent empirical papers and note exactly how they report effect sizes. That is your template.

How to Write About Effect Size in Your Results Section

Reporting effect size is not just about including a number. It is about interpreting that number in a way that helps the reader understand what your result means. A number without context is almost as unhelpful as no number at all. Your results section should report the effect size statistic, its confidence interval where possible, and a brief interpretation of its magnitude.

A well-written results sentence might look like this: participants in the intervention group scored significantly higher than controls, t(48) = 3.12, p = .003, d = 0.88, 95 percent CI [0.35, 1.41], indicating a large effect. That single sentence gives a reviewer everything they need: the test statistic, the p-value, the effect size, and the uncertainty around it. Nothing is hidden. Nothing requires the reviewer to go searching.

In your discussion section, return to the effect size and explain what it means for your research question. If the effect is small, acknowledge it and explain whether that is expected given the field, or whether it limits the practical implications of your work. Reviewers respect honesty about limitations far more than they respect silence about them. Pretending a small effect is large is one of the fastest ways to receive a rejection.

Reading about what peer-reviewed research actually requires will sharpen your instincts here. The post on what is peer-reviewed research and why it matters covers the standards your paper will be held to.

Why Effect Size Matters More for Student Researchers Than You Think

Student researchers often work with smaller samples than professional researchers. A high school student running a psychology study might have access to 30 or 40 participants. A college student in an independent research program might have 60. These sample sizes make it harder to reach statistical significance, which can feel discouraging. But effect size reframes the situation.

A study with 35 participants that finds a large effect size, say a Cohen's d of 0.9, is genuinely interesting. It suggests a strong signal worth investigating further with a larger sample. Reviewers at journals that publish student research, including Journal of Emerging Investigators and the International Journal of High School Research, understand that student studies are often exploratory. They are not expecting nationally representative samples. They are expecting honest, careful methodology and transparent reporting.

Reporting effect size in a small-sample study is actually a sign of methodological maturity. It shows you understand the difference between what your data can and cannot claim. That is the kind of intellectual honesty that gets papers accepted. For more on how specific journals for student researchers approach these standards, the guide to the International Journal of High School Research and what it publishes is worth reading before you submit.

Publication Compass is a platform designed to help student researchers navigate exactly these kinds of methodological expectations. It provides structured feedback on drafts and helps identify the right journals for your work, so you are not guessing about what reviewers want.

Frequently Asked Questions

What is effect size in simple terms?

Effect size is a number that tells you how large or meaningful a result is. It goes beyond asking whether an effect exists and tells you how strong it is. A small effect size means the result is real but modest. A large effect size means the result is substantial and likely to matter in practice.

Why do reviewers care about effect size and why reviewers care about it so strongly?

Reviewers care because statistical significance alone can be misleading. With a large enough sample, even a trivial difference becomes significant. Effect size gives reviewers a way to judge whether a result is practically meaningful, not just mathematically detectable. Journals increasingly require it as a standard part of results reporting.

Do I need to report effect size if my result is not significant?

Yes, and in some ways it matters more when results are not significant. A non-significant result with a large effect size suggests the study may have been underpowered, meaning the sample was too small to detect a real effect. Reporting this helps readers interpret your findings correctly and guides future researchers.

Which effect size measure should I use for my study?

The right measure depends on your design. Use Cohen's d when comparing two group means. Use Pearson's r for correlations. Use eta-squared for ANOVA designs. When in doubt, look at three or four recent papers in your target journal and follow the conventions they use. Your field's norms matter more than any general rule.

Can I calculate effect size after collecting my data?

Yes. Effect size is calculated from your data, so you compute it after data collection alongside your other statistics. Most statistical software, including R, SPSS, and jamovi, can calculate common effect sizes directly. If you are calculating by hand, Cohen's d for two groups is the difference between the means divided by the pooled standard deviation.

Getting This Right Before You Submit

Effect size is not a technicality. It is a core part of what makes a result meaningful. Reviewers at serious journals have been asking for it for decades, and the expectation is only growing stronger as the field moves away from over-reliance on p-values. Getting comfortable with effect size reporting now, before your first submission, puts you ahead of most student researchers who encounter it for the first time in a rejection letter.

The practical steps are straightforward: choose the right measure for your design, calculate it alongside your significance tests, report it with a confidence interval, and interpret it honestly in your discussion. If your effect is small, say so and explain what that means for your conclusions. That transparency is what reviewers are looking for. For more guidance on the full research and publication process, explore the Publication Compass blog, where each post is written to help student researchers move from draft to submission with confidence.