A t test is a statistical method used to compare averages.
Despite its technical name, its main purpose is usually quite simple.
A t test helps you decide whether a difference between two averages is large enough to provide evidence of a real difference, rather than simply reflecting the natural variation that occurs in sample data.
Imagine you want to compare the final examination scores of students taught using two different teaching methods.
You have 80 students:
The average examination scores are:
The classroom students therefore score 5.10 points higher on average.
At first, you might conclude that classroom teaching produced better results.
However, the averages do not tell the whole story.
Students taught in the same way will not all receive exactly the same examination score. Some may score much higher than the average and others much lower.
The classroom group has a standard deviation of 8.60, and the online group has a standard deviation of 9.40.
The t test takes this variation into account when assessing the difference between the two averages.
For these data, the result is t(78) = 2.53, p = .013.
This provides statistical evidence that the average examination scores differ between the two groups.
Suppose two groups have average scores of 74 and 69.
The difference is 5 points.
Whether that difference provides convincing evidence depends partly on how much the individual scores vary.
If almost all the students in each group score close to their group average, a 5-point difference may be quite noticeable.
If the scores within each group vary widely, the same 5-point difference may be less convincing.
Sample size also matters.
A difference based on 200 students generally provides more information than the same difference based on six students.
A t test therefore considers:
These are combined to produce the t statistic.
The t statistic compares the observed difference with the amount of uncertainty surrounding that difference.
In simple terms, t = observed difference ÷ standard error of the difference
The standard error measures how much the difference between sample averages would be expected to vary from sample to sample.
In the teaching-method example, t = 2.53.
This means that the observed difference is about 2.53 standard errors away from the value expected if there were no difference between the population averages.
A t value close to zero indicates that the observed difference is small relative to the uncertainty in the data.
As the absolute t value becomes larger, the statistical evidence for a difference generally becomes stronger.
The positive or negative sign shows the direction of the difference. The sign depends on which group is subtracted from the other.
Understanding t(78) = 2.53
A t test result is commonly written in the form t(78) = 2.53.
The value 2.53 is the t statistic.
The value 78 in parentheses is the degrees of freedom, usually written as df.
For a standard independent-samples t test with two groups, df = n₁ + n₂ − 2.
There are 40 students in each group in this example:
40 + 40 − 2 = 78
The degrees of freedom help determine which t distribution should be used when calculating the p value.
You do not normally need to calculate the degrees of freedom yourself. Statistical software does this as part of the analysis.
The p value helps you assess the statistical evidence for a difference.
In the teaching-method example, p = .013.
This means that if there were actually no difference between the population averages, a difference producing a t statistic this extreme or more extreme would occur about 1.3% of the time through random sampling variation.
A result is commonly treated as statistically significant when p < .05.
Because .013 is below .05, the difference between the classroom and online groups is statistically significant.
This provides evidence that the difference is not simply the result of ordinary sampling variation.
However, statistical significance does not tell you whether the difference is large or important.
That requires you to consider the size of the difference as well.
An effect size describes the size of a difference.
A commonly reported effect size for a t test is Cohen's d.
In the teaching-method example, Cohen's d = 0.57.
Cohen's d expresses the difference between the groups in standard deviation units.
As a rough guide:
These are guidelines rather than strict rules. The importance of an effect depends on what is being studied.
A difference of only a few points may be important in one situation but unimportant in another.
In the teaching-method example, d = 0.57 indicates a moderate difference between the two groups.
The p value and Cohen's d therefore answer different questions:
A result can be statistically significant but have only a small effect size, particularly when the sample is large.
An independent-samples t test compares the averages of two separate groups.
The students in one group are different people from those in the other group.
The teaching-method example is an independent-samples t test because:
Each student belongs to only one group.
Other examples include comparing:
The important point is that the observations in one group are independent of those in the other group.
A paired-samples t test compares two related sets of scores.
Often, the same participants are measured twice.
Suppose 80 students complete an examination before and after taking part in a study-skills program.
Their average scores are:
The average score increased by 5.20 points.
The t test result is t(79) = 6.20, p < .001 and Cohen's d = 0.69.
The result provides strong statistical evidence that the average scores changed between the two measurements.
The paired-samples t test works by considering the change for each student.
For example, if one student's score increases from 65 to 72, that student's change is 7 points.
The analysis examines these individual differences and determines whether their average differs significantly from zero.
This is why the paired test is appropriate when the same participants are measured before and after an intervention.
For a paired-samples t test, df = N − 1.
There are 80 students in the example, so df = 80 − 1 = 79.
This explains the result t(79) = 6.20.
The number of measurements is not 160 for this purpose. There are 80 pairs of scores, one pair for each student.
The choice depends on how the data were collected.
Use an independent-samples t test when you compare two separate groups.
Use a paired-samples t test when the observations are naturally linked, such as:
Do not use an independent-samples test for paired data.
The connection between the two measurements contains useful information that the paired test takes into account.
A third type is the one-sample t test.
Instead of comparing two groups, it compares the average of one sample with a specified value.
For example, suppose the established average examination score is 70.
You collect scores from a new group of students and find that their average is 74.
A one-sample t test can help you determine whether the difference between 74 and 70 provides statistical evidence that the population average for the new group differs from 70.
The three common types can therefore be summarized as:
| Question | Appropriate test |
|---|---|
| Are the averages of two separate groups different? | Independent-samples t test |
| Did the average change for the same participants? | Paired-samples t test |
| Does one sample average differ from a specified value? | One-sample t test |
A t test begins with the idea that there is no population difference.
This is called the null hypothesis.
For the teaching-method example, the null hypothesis states that the population mean examination score is the same for classroom and online teaching.
The sample averages ( 74.20 versus 69.10) are different.
The t test asks whether a difference of this size would be unusual if the null hypothesis were true.
The p value helps answer that question.
A small p value provides evidence against the null hypothesis.
It does not tell you the probability that the null hypothesis itself is true or false.
Sample size affects how precisely you can estimate a difference.
Suppose two groups differ by 5 points.
If each group contains only five students, the result may be quite uncertain.
If each group contains 500 students, the same difference can be estimated much more precisely.
This means that a small difference can sometimes be statistically significant in a very large sample.
A larger difference may fail to reach statistical significance in a very small sample.
For this reason, you should consider both:
Do not use the p value alone to judge how important a difference is.
A t test makes assumptions about the data.
The exact assumptions depend on the type of t test, but some important considerations include the following.
The outcome being compared should normally be numerical.
Examination scores, height, income, reaction time, and blood pressure are examples.
A t test is not normally used to compare categories such as preferred teaching method or employment status.
For an independent-samples t test, observations should be independent.
A student in the classroom group should not also appear in the online group.
If the observations are deliberately related, a paired-samples test may be more appropriate.
The t test assumes that the relevant population distribution is approximately normal.
For a paired-samples test, it is the distribution of the differences between the paired scores that is particularly important.
The t test can often tolerate moderate departures from normality, particularly with larger samples, but strong skew or extreme outliers can affect the results.
Examining the data before running the test is therefore important.
The traditional independent-samples t test assumes that the two populations have equal variances.
This means that the amount of variation in one population is assumed to be similar to the amount in the other.
When this assumption is not appropriate, a version called Welch's t test can be used.
Welch's t test also compares the averages of two independent groups.
Unlike the traditional Student's independent-samples t test, it does not require the two populations to have equal variances.
Statistical software can calculate Welch's test automatically.
One visible difference is that its degrees of freedom may contain a decimal.
For example, you might see t(67.42) = 2.31
The decimal degrees of freedom are not an error. They result from the adjustment used by Welch's test.
The interpretation of the t and p values remains broadly the same.
A t test and ANOVA both examine differences between averages.
A t test is particularly useful when you want to compare two averages.
ANOVA is commonly used when you want to compare three or more groups or examine more complicated designs involving several factors.
For example:
When there are only two independent groups, a conventional two-sided t test and a one-way ANOVA test the same basic difference and lead to the same statistical-significance conclusion.
The t test is usually simpler when only two averages need to be compared.
A statistically significant t test does not automatically mean that the difference is important.
For example, a very large study could identify a difference of only one examination point as statistically significant.
Whether that one point matters is a separate question.
A t test also does not by itself establish cause and effect.
If classroom students score higher than online students, the teaching method may not be the only difference between the groups.
The groups might also differ in:
Whether a causal conclusion is justified depends on how the study was designed.
A non-significant result also does not prove that the two population averages are exactly the same.
It means that the study did not provide sufficient statistical evidence of a difference.
Before interpreting the result, it helps to see where the numbers come from.
For the teaching-method example, the original data can be summarized as follows:
| Teaching method | n | M | SD |
|---|---|---|---|
| Classroom | 40 | 74.20 | 8.60 |
| Online | 40 | 69.10 | 9.40 |
The t test uses these values to calculate the test results.
The difference between the two averages is 74.20 − 69.10 = 5.10.
The 78 degrees of freedom come from the sample sizes, that is, 40 + 40 − 2 = 78.
The difference between the averages, the standard deviations, and the sample sizes are used to calculate the t statistic of 2.53.
The t statistic and the degrees of freedom are then used to calculate the p value of .013.
Cohen's d of 0.57 is calculated from the difference between the two averages relative to the variation in the scores.
Consider the teaching-method result again:
t(78) = 2.53, p = .013, Cohen's d = 0.57.
The complete result is therefore t(78) = 2.53, p = .013, and Cohen's d = 0.57.
In simple terms, students taught in the classroom scored higher on average than students taught online.
The classroom students scored 5.10 points higher on average.
The p value of .013 indicates that this difference is statistically significant, so it is unlikely to be due to random variation alone.
Cohen's d of 0.57 indicates that the difference between the two groups is moderate in size.
For this study, the classroom teaching method was associated with better exam results than the online method.
Whether the teaching method itself caused the difference depends on how the study was designed.
In practice, statistical software performs these calculations. Programs such as IBM SPSS Statistics and the free alternative jamovi can calculate the statistics needed for the analysis and present the results in a form you can interpret and report.
You do not need to calculate the t statistic, degrees of freedom, or p value by hand to understand the analysis.
The important part is understanding:
For examples showing how to present the results, see t Test Table in APA Format.
A t test compares averages and helps you assess whether an observed difference provides statistical evidence of a difference in the population.
An independent-samples t test compares two separate groups.
A paired-samples t test compares related measurements, often from the same participants.
A one-sample t test compares one sample average with a specified value.
The t statistic compares the observed difference with the uncertainty surrounding that difference.
The degrees of freedom help determine the appropriate t distribution, and the p value helps you assess statistical significance.
Cohen's d can be used to describe the size of the difference.
Statistical significance and effect size answer different questions. A significant result is not necessarily an important result, and a t test does not by itself establish cause and effect.
Why can't I just compare the two averages?
The averages show the observed difference but not how much uncertainty surrounds it.
A difference of 5 points may be convincing when individual scores are tightly grouped but much less convincing when the scores vary widely.
A t test takes the variation and sample size into account.
Can a very small difference be statistically significant?
Yes.
With a sufficiently large sample, even a small difference can produce a small p value.
That is why it is useful to consider an effect size such as Cohen's d as well as statistical significance.
Is p = .049 meaningfully different from p = .051?
Not necessarily.
Using .05 as a cutoff would label the first result statistically significant and the second not statistically significant, but the two p values provide very similar evidence.
It is usually better to report the actual p value and consider the size and uncertainty of the effect rather than treating .05 as a sharp dividing line between an important and an unimportant result.
Why not run several t tests when I have three or more groups?
Running many separate t tests increases the chance of finding at least one statistically significant result simply through random variation.
ANOVA is usually a better starting point when you want to compare three or more group averages.
Further comparisons can then be made using appropriate follow-up tests.
What happens if the two groups have very different standard deviations?
A traditional independent-samples t test assumes equal population variances.
If this assumption is doubtful, Welch's t test is often appropriate because it allows the variances to differ.
Its degrees of freedom may be reported as a decimal.
Does a non-significant result mean the groups are the same?
No.
A non-significant result means that the analysis did not provide sufficient statistical evidence of a difference.
There may genuinely be little or no difference, or the study may simply have been too small or too variable to detect one clearly.
Can a t test prove that one treatment caused the difference?
Not by itself.
A t test tells you whether there is statistical evidence of a difference between averages.
Whether one treatment caused that difference depends on the study design, particularly how participants were assigned to groups and whether other explanations have been controlled.