Common statistical mistakes in student papers

TL;DR

• Misreading p-values is the most frequent statistical error in student research.

• Correlation does not imply causation. Confusing the two weakens your argument.

• Sample size affects every statistical conclusion you draw.

• Choosing the wrong test for your data type invalidates your results.

• Reporting raw numbers without effect sizes tells reviewers very little.

You have spent weeks collecting data. You have run your analysis. You are confident in your findings. Then a peer reviewer sends back a rejection, and the reason is not your research question or your literature review. It is your statistics.

This is one of the most common experiences for first-time student researchers. The experimental design is solid. The writing is clear. But the statistical analysis contains errors that undermine the entire paper. Reviewers at journals like the Journal of Student Research and PLOS ONE flag these problems consistently, and many submissions never recover from them.

Understanding where student papers go wrong statistically is not about memorising formulas. It is about knowing what reviewers are looking for and what your numbers are actually saying. This post walks through the most common statistical mistakes in student papers, explains why each one matters, and shows you how to fix them before submission.

Why Statistical Errors Are So Costly in Peer Review

Statistical errors in student papers are costly because they signal to reviewers that the conclusions cannot be trusted, even if the underlying data is sound. A single misused test or a misreported p-value can lead to outright rejection, regardless of how interesting the research question is. Reviewers are trained to check statistical methods first, because errors there invalidate everything that follows.

Peer-reviewed journals operate on the assumption that reported results are reproducible. When a reviewer sees a p-value misinterpreted, or a sample of twelve participants used to make sweeping generalisations, they cannot recommend publication. The problem is not that students are careless. It is that statistical reasoning is rarely taught in depth before students attempt their first independent paper.

Most high school and early undergraduate curricula cover descriptive statistics: means, medians, standard deviations. Inferential statistics, the kind that tells you whether your results mean something beyond your sample, gets far less attention. That gap is where most of the common statistical mistakes in student papers originate.

If you are preparing your first submission and want structured guidance on the full process, the overview at how to publish a research paper as a high school student is a useful starting point before diving into methodology.

Misunderstanding What a P-Value Actually Means

A p-value tells you the probability of observing your results, or more extreme results, if the null hypothesis were true. It does not tell you the probability that your hypothesis is correct. A p-value below 0.05 does not mean your finding is important. It means your result would be unlikely to occur by chance alone, under specific assumptions.

This distinction matters enormously. Student papers frequently state something like: "the p-value was 0.03, which proves our hypothesis." That sentence contains two errors. First, statistics do not prove hypotheses. Second, a p-value of 0.03 only tells you about the probability of your data given the null hypothesis, not about the truth of your theory.

The American Statistical Association published a statement in 2016 clarifying the correct interpretation of p-values, specifically because misuse had become widespread across scientific literature. If professional researchers get this wrong, it is unsurprising that student papers do too. The fix is straightforward: report your p-value, state whether it crosses your pre-specified threshold, and then separately discuss what the finding means in context.

If you are working through your first paper and want to avoid the broader category of errors that trip up new researchers, the post on common mistakes first-time researchers make covers the wider landscape beyond statistics alone.

Confusing Correlation with Causation

Correlation measures the relationship between two variables. Causation means one variable directly causes a change in another. These are not the same thing, and treating them as equivalent is one of the most common statistical mistakes in student papers. Finding that two variables move together does not tell you why, or which one influences the other, or whether a third variable is driving both.

A student paper might find a strong positive correlation between hours of sleep and exam scores in a sample of 80 students. That is a legitimate finding. But concluding that sleeping more causes higher exam scores goes beyond what the data supports. A third variable, such as lower stress levels, could independently produce both outcomes.

The language you use in your results and discussion sections carries real weight. Words like "leads to," "causes," and "results in" imply causation. If your study is observational or correlational, use language that reflects that. "Is associated with," "predicts," and "correlates with" are more accurate and will not draw criticism from reviewers.

Establishing causation requires either a randomised controlled experiment or a specific set of statistical techniques designed for causal inference. If your study design does not support causal claims, do not make them. Reviewers will notice, and it is one of the faster ways to receive a rejection.

Using Too Small a Sample and Not Acknowledging It

A small sample size does not automatically invalidate a study, but failing to acknowledge its limitations does. Student research often involves samples of 20 to 50 participants because of practical constraints, and that is understandable. The error is not the small sample itself. It is drawing broad conclusions from it without discussing what that means for the generalisability of the findings.

Statistical power is the ability of a test to detect a real effect when one exists. Small samples reduce statistical power. This means that even if a genuine effect is present in the population, your study may not detect it. Conversely, a small sample can produce a statistically significant result that disappears when tested on a larger group.

Before running your analysis, it is worth doing a power calculation. Tools like G*Power, which is free and widely used in academic research, allow you to estimate the minimum sample size needed to detect an effect of a given size at a specified confidence level. If your sample falls short of that estimate, say so explicitly in your limitations section. Reviewers respect transparency far more than they respect silence on a known weakness.

If you are planning to submit to a journal and want to understand how journals evaluate methodology, the guide on Journal of Student Research scope requirements and submission explains what reviewers at student-focused publications prioritise.

Choosing the Wrong Statistical Test

Choosing the wrong statistical test is one of the most common statistical mistakes in student papers, and it often goes undetected by the student because the software still produces an output. The output just does not mean what the student thinks it means. Each statistical test carries assumptions about the type of data being analysed, the distribution of that data, and the relationship between groups being compared.

Here is a basic framework for matching tests to data types:

1. Comparing two independent groups on a continuous variable: use an independent samples t-test, provided the data is approximately normally distributed.

2. Comparing more than two groups: use a one-way analysis of variance (ANOVA), not multiple t-tests. Running multiple t-tests inflates your Type I error rate.

3. Examining the relationship between two continuous variables: use Pearson correlation if the data is normally distributed, or Spearman rank correlation if it is not.

4. Analysing categorical data, such as yes/no responses or group membership: use a chi-square test, not a t-test.

5. Comparing the same participants across two time points: use a paired samples t-test, not an independent samples t-test.

This is not an exhaustive list, but it covers the tests that appear most frequently in student papers and the errors that accompany them. When in doubt, consult a statistics textbook or a methodology guide specific to your field before running your analysis.

If you are preparing a paper that involves technical methodology and are considering conference submission, the post on IEEE conference papers and student submission covers what reviewers in technical fields expect from your methods section.

Reporting Results Without Effect Sizes

A statistically significant result tells you that an effect probably exists. An effect size tells you how large that effect is. Reporting one without the other gives reviewers an incomplete picture. This is a consistent gap in student papers, and it is increasingly flagged by journals as a reason for revision or rejection.

Consider two studies. Both find a statistically significant difference in test scores between two groups. Study A has an effect size of 0.8 (large, by Cohen's conventions). Study B has an effect size of 0.1 (small). The p-values might look identical, but the practical implications are entirely different. Study A found something that matters in the real world. Study B found something that is statistically detectable but possibly trivial.

Common effect size measures include Cohen's d for comparing means, Pearson's r for correlations, and eta-squared for ANOVA results. Most statistical software packages calculate these automatically. Report them alongside your p-values, and briefly interpret what the size means in the context of your research question. This single addition can significantly strengthen the credibility of your results section.

Publication Compass provides structured feedback on student manuscripts, including the methodology and results sections, so you can identify gaps like missing effect sizes before a reviewer does. If you are close to submission, joining the waitlist at publicationcompass.ai gives you early access to that feedback process.

Presenting Data Without Checking Assumptions

Every parametric statistical test rests on assumptions about the data. The t-test assumes approximate normality. Pearson correlation assumes a linear relationship. ANOVA assumes homogeneity of variance across groups. Running these tests without checking whether your data meets those assumptions is one of the quieter but more damaging common statistical mistakes in student papers.

Checking assumptions is a step that many students skip because it adds time and because the software does not force you to do it. But a reviewer who looks at your data and notices a heavily skewed distribution in a t-test analysis will question the validity of every finding that follows.

The process for checking assumptions is straightforward:

1. Test for normality using a Shapiro-Wilk test (recommended for samples under 50) or a Kolmogorov-Smirnov test for larger samples.

2. Check for homogeneity of variance using Levene's test before running an ANOVA or independent t-test.

3. Plot your data. Histograms, Q-Q plots, and scatterplots reveal distribution problems that numerical tests sometimes miss.

If your data violates the assumptions of a parametric test, use the non-parametric equivalent. The Mann-Whitney U test replaces the independent t-test. The Kruskal-Wallis test replaces ANOVA. These alternatives are not inferior. They are simply more appropriate when your data does not meet parametric requirements.

Frequently Asked Questions

What are the most common statistical mistakes in student papers?

The most common statistical mistakes in student papers include misinterpreting p-values as proof of a hypothesis, confusing correlation with causation, using underpowered samples without acknowledging limitations, selecting the wrong statistical test for the data type, and omitting effect sizes from results. Each of these errors can lead to peer review rejection, even when the underlying research is well-designed.

How do I know which statistical test to use in my paper?

The right statistical test depends on your data type, the number of groups you are comparing, and whether your data meets parametric assumptions. Continuous data comparing two groups typically uses a t-test. Categorical data uses a chi-square test. More than two groups require ANOVA. Always check your data's distribution before selecting a test, and use non-parametric alternatives when assumptions are violated.

Does a p-value below 0.05 mean my results are correct?

No. A p-value below 0.05 means your results would be unlikely to occur by chance if the null hypothesis were true. It does not confirm your hypothesis, and it does not measure the size or importance of an effect. The American Statistical Association has explicitly cautioned against treating the 0.05 threshold as a binary indicator of truth. Always pair p-values with effect sizes and contextual interpretation.

How small is too small for a sample size in student research?

There is no universal minimum, but sample size should be determined by a power calculation before data collection begins. A sample that is too small to detect the effect you are studying produces unreliable results, whether significant or not. Tools like G*Power can calculate the minimum sample needed based on your expected effect size, alpha level, and desired statistical power, typically set at 0.80.

Should I report effect sizes even if my result is not statistically significant?

Yes. A non-significant result with a large effect size suggests your study may have been underpowered, which is important information for future researchers. A non-significant result with a small effect size suggests the effect may genuinely be negligible. Reporting effect sizes in both cases gives your results more meaning and demonstrates statistical literacy to reviewers.

Getting Your Statistics Right Before Submission

Statistical errors are fixable. The key is catching them before a reviewer does. Work through your assumptions before running any test. Report effect sizes alongside p-values. Be precise about what your data can and cannot support. If your study is correlational, say so clearly and let your language reflect that throughout the paper.

The difference between a paper that gets rejected and one that moves forward to peer review is often not the research question. It is the rigour of the analysis and the honesty of the reporting. Take the time to get those right. For more guidance on the full submission process and what journals expect at each stage, the how to publish a research paper as a student guide covers the process from draft to decision.