What is a t-test and when do you need one

TL;DR: This post answers what a t-test is, when to use one, and how to apply it correctly in a high school research project. It is written for students in grades 9 through 12 who are working with numerical data and need to compare group means. After reading, you will know which type of t-test fits your study design and how to interpret the result. Students whose research is ready for peer review can submit their work to the Princeton Journal of Pre-Collegiate Research.

Introduction

The most common statistical error in student research papers is not a miscalculation. It is choosing the wrong test entirely. A t-test is one of the most frequently used statistical tools in student research, and one of the most frequently misapplied. Understanding what is a t-test and when do you need one is essential before any data analysis begins. This post explains the t-test precisely, identifies the three main types, and gives you a clear decision framework for choosing the right one. It also covers the mistakes that cause reviewers to reject papers at the analysis stage.

What is a t-test and when do you need one?

A t-test is a statistical test used to determine whether the means of two groups are significantly different from each other. It produces a t-statistic and a p-value. If the p-value falls below a pre-set threshold, typically 0.05, the difference between the groups is considered statistically significant. You need a t-test when your outcome variable is numerical, you are comparing exactly two means, and your data is approximately normally distributed.

The t-test was developed by statistician William Sealy Gosset in 1908 and published under the pseudonym "Student," which is why it is also called the Student's t-test. It remains one of the foundational tools in quantitative research across biology, psychology, economics, and the social sciences.

There are three types of t-test, and choosing the correct one depends entirely on your study design.

1. One-sample t-test: Compares the mean of a single group against a known or hypothesised value. Example: testing whether the average sleep duration of students in your school differs from the nationally reported average of 7 hours.

2. Independent samples t-test: Compares the means of two separate, unrelated groups. Example: comparing test scores between students who studied with music and students who studied in silence. This is the most common type in student research.

3. Paired samples t-test: Compares means from the same group measured at two different points in time, or under two different conditions. Example: measuring reaction time in the same participants before and after a caffeine intervention.

The decision between these three types is not flexible. Using an independent samples t-test when a paired design is appropriate, or vice versa, produces invalid results. Reviewers check this. Before running any analysis, identify whether your two sets of measurements come from the same participants or different participants. If the same: use a paired t-test. If different: use an independent samples t-test.

A t-test also requires that certain assumptions are met. The data should be continuous and numerical, not categorical. The distribution of values should be approximately normal, which can be checked with a Shapiro-Wilk test for small samples. For the independent samples t-test, the variance in each group should be roughly equal, a condition assessed with Levene's test. If these assumptions are violated, a non-parametric alternative such as the Mann-Whitney U test may be more appropriate. Understanding what a p-value actually means in this context is also essential; see the detailed explanation in this post on p-values for high school researchers.

What do t-test results actually tell you, and what do they not?

A statistically significant t-test result tells you that the observed difference between two group means is unlikely to have occurred by chance alone, given the sample size and variability in your data. It does not tell you that the difference is large, meaningful, or practically important.

This distinction matters enormously in peer review. A study with a very large sample size can produce a statistically significant result for a difference so small it has no real-world relevance. Conversely, a study with a small sample may fail to reach significance even when a meaningful difference exists, simply because the test lacked statistical power.

This is why reviewers increasingly expect students to report effect size alongside the t-statistic and p-value. Effect size measures the magnitude of the difference, independent of sample size. For t-tests, Cohen's d is the standard effect size measure. A Cohen's d of 0.2 is considered small, 0.5 is medium, and 0.8 is large, according to conventions established by statistician Jacob Cohen. Reporting only a p-value without effect size is now considered incomplete in most peer-reviewed contexts. For a full explanation of how to calculate and interpret effect size, see this guide on effect size and why reviewers care about it.

The t-test also does not establish causation. If students who reported higher sleep hours also scored higher on a cognitive task, a t-test can confirm the groups differ significantly. It cannot confirm that sleep caused the improvement. Causal claims require experimental designs with random assignment, not just statistically significant comparisons.

What are the most common t-test mistakes high school researchers make?

The most common t-test mistakes in student research fall into four categories, and each one is detectable by a peer reviewer in under two minutes.

Mistake 1: Using a t-test to compare more than two groups. A t-test compares exactly two means. Students who compare three or more groups, such as a control group and two treatment conditions, sometimes run multiple t-tests instead of using an ANOVA (analysis of variance). Running three separate t-tests across three groups inflates the probability of a false positive result. This is called the multiple comparisons problem. The fix is to use a one-way ANOVA when comparing three or more group means, then apply post-hoc tests if needed.

Mistake 2: Ignoring the assumption of normality. T-tests assume that the data within each group is approximately normally distributed. With very small samples, fewer than 15 per group, this assumption is difficult to verify and easy to violate. Students often skip the normality check entirely. The consequence is that the p-value produced may be inaccurate. The fix is to run a Shapiro-Wilk test on each group before conducting the t-test, and to switch to a Mann-Whitney U test if normality is rejected.

Mistake 3: Confusing statistical significance with practical significance. A p-value below 0.05 does not mean the finding is important. It means the result is unlikely to be due to chance. Students frequently write conclusions that overstate what significance means. The fix is to always report Cohen's d alongside the p-value and to interpret the effect size explicitly in the discussion section.

Mistake 4: Choosing the wrong t-test type for the study design. Using an independent samples t-test when the data is paired, or a paired t-test when the groups are independent, produces incorrect degrees of freedom and an invalid result. The fix is to confirm, before analysis, whether the two sets of measurements come from the same participants or different ones. That single question determines the correct test.

How to apply a t-test correctly in your research, step by step

1. Define your research question in terms of two group means. Confirm that your outcome variable is numerical and continuous, not categorical.

2. Identify your study design. Are the two groups made up of different participants, or are they the same participants measured twice? Different participants: independent samples t-test. Same participants: paired samples t-test. One group compared to a known value: one-sample t-test.

3. Check the normality assumption. Run a Shapiro-Wilk test on each group. If the p-value from that test is above 0.05, normality is not rejected and you can proceed. If it is below 0.05, consider a non-parametric alternative.

4. For independent samples: check equal variance. Run Levene's test. If variance is significantly unequal, use Welch's t-test, which does not assume equal variances. Most statistical software applies Welch's correction automatically.

5. Run the t-test and record the t-statistic, degrees of freedom, and p-value. Do not round the p-value to simply "significant" or "not significant." Report the exact value.

6. Calculate and report Cohen's d. This is not optional in a publishable paper. The formula for Cohen's d in an independent samples design is the difference between the two means divided by the pooled standard deviation. For a clear explanation of standard deviation in this context, see this guide on standard deviation and why it matters.

7. Write up the results in APA format. A correctly formatted t-test result reads as follows: t(28) = 3.14, p = .003, d = 0.72. The number in parentheses is the degrees of freedom. This format is expected in most peer-reviewed social science and psychology journals.

8. Submit your completed paper for peer review. Review the submission guidelines at princeton-jpcr.org/submit before preparing your manuscript.

PJPCR publishes original quantitative research across all academic disciplines, including studies that use t-tests and other inferential statistics. If your data analysis is complete and your paper is ready for peer review, review the submission guidelines at princeton-jpcr.org/submit.

Frequently asked questions about t-tests in student research

What is a t-test in simple terms?

A t-test is a statistical test that determines whether the average values of two groups are different enough to be considered statistically significant rather than the result of random variation. It produces a p-value, which indicates the probability of observing the measured difference if no real difference existed. T-tests are used across biology, psychology, economics, and many other fields where numerical comparisons between two groups are needed.

How long does it take to run a t-test, and what software do students use?

Running a t-test takes under five minutes once your data is organised. Most students use Excel, Google Sheets, JASP (free and open-source), or SPSS. JASP is particularly recommended for student researchers because it produces output in APA format automatically and includes effect size calculations by default. Organising your data correctly, with one column per group or one column per time point, is the step that takes the most time.

Do I need a university lab or mentor to use a t-test in my research?

No. A t-test requires only a spreadsheet or free statistical software, a clearly defined research question, and data collected through a sound methodology. Many high school students conduct independent t-test analyses using survey data, publicly available datasets, or data collected through school-based experiments. Mentorship is valuable but not a prerequisite for conducting valid inferential statistics at this level.

What makes a t-test result publishable in a peer-reviewed journal?

A publishable t-test result is one where the assumptions have been checked and reported, the correct test type was selected for the study design, effect size is reported alongside the p-value, and the interpretation does not overstate what the statistic shows. Reviewers reject papers that report only p-values, that fail to address assumption violations, or that conflate statistical significance with causal claims. Transparent reporting of all results, including non-significant ones, is also expected.

What kinds of research that use t-tests does PJPCR publish?

The Princeton Journal of Pre-Collegiate Research publishes original quantitative research across the sciences, social sciences, and interdisciplinary fields. Studies using t-tests appear regularly in submissions covering psychology, public health, environmental science, and economics. All submissions undergo peer review conducted by qualified reviewers. The standard review timeline is 2 to 3 months, and a fast-track option is available for students who need a quicker turnaround. Full details are available on the peer review process page.

Conclusion

A t-test is a precise tool with specific requirements. It compares exactly two group means, requires numerical data, and depends on assumptions about normality and variance that must be checked before the test is run. Choosing the correct type, one-sample, independent samples, or paired, is determined by the structure of your data, not by preference. Reporting effect size alongside the p-value is not optional in peer-reviewed research; it is expected. Students who apply these principles correctly produce analyses that hold up under scrutiny.

If your research uses a t-test and the analysis is complete, the next step is preparing your manuscript for submission. If your work is ready for peer review, submit it to PJPCR at princeton-jpcr.org/submit.