Null hypothesis: People’s ratings of trust towards strangers are identical regardless of whether they were exposed to oxytocin or not.

Alternative hypothesis: Compared to people who are not exposed to oxytocin, people who are exposed to oxytocin will give a higher rating of trust towards strangers.

The confidence interval for the oxytocin group ranges from 6 to 10 so has a length of 4 and an MOE of half this value (i.e., 2). For the non-oxytocin group, it ranges from 4.5 to 7.5, which is a length of 3 and an MOE of 1.5. The average MOE is, therefore, (2 + 1.5)/2 = 1.75. Moderate overlap, which would indicate a significant difference between groups at p = 0.05, would be half of this value (i.e., 0.88). We can calculate the size of the overlap of the confidence intervals by subtracting the lower limit of the confidence interval for the oxytocin group (6) from the upper limit of the confidence interval for the non-oxytocin group (7.5), which gives us 1.5. The moderate overlap (0.88) tells us the maximum amount of overlap that would yield a significant difference (assuming a 0.05 Type I error rate); therefore, an overlap less than this indicates that p would be less than 0.05 (i.e., a difference deemed more significant than p < 0.05), whereas an overlap greater than this value indicates that p is greater than 0.05 (which is typically interpreted as non-significant). In this example, therefore, because the overlap of 1.5 is greater than 0.88 (the overlap for p = 0.05), it suggests that there is more overlap than you would get for p < 0.05; therefore, the oxytocin and non-oxytocin groups are unlikely to be significantly different from each other using an α-level of 0.05.

A Type I error is when you believe that there is a genuine effect in the population, when in fact there is not. A Type II error is the opposite: when you believe that there is no effect in the population when, in reality, there is.

1. It doesn’t allow you to draw any firm conclusions about the null hypotheses.
    
2. Significance is affected by the sample size.
    
3. It encourages all-or-nothing thinking.
    
4. It depends on the intentions of the scientist.

The significance of a test statistic is directly linked to the sample size: the same effect will have different p-values in different-sized samples: small differences can be deemed ‘significant’ in large samples, and large effects might be deemed ‘non-significant’ in small samples.

If you do a one-tailed test and the results turn out to be in the opposite direction to what you predicted you must ignore them and accept the null hypothesis. If you don’t do this, then you have done a two-tailed test using a different level of significance from the one you set out to use, which is cheating; therefore, one-tailed tests temp people to cheat. Also, if you do a two-tailed test and find that your p is 0.06, then you would conclude that your results were not significant (because 0.06 is bigger than the critical value of 0.05). Had you done this test one-tailed, however, the p you would have got would be half of the two-tailed value (0.03). This one-tailed value would be significant at the conventional level (because 0.03 is less than 0.05). Therefore, a scientist who finds a two-tailed p that is just non-significant might be tempted to pretend that they had always intended to do a one-tailed test because the ‘one-tailed’ p-value is significant.

Null hypothesis: Album sales are identical regardless of whether or not you give away a free gift.

Alternative hypothesis: More albums are sold when accompanied by a free gift.

This hypothesis is directional, since it states that more albums will be sold if a free gift is given. Importantly, a result in the opposite direction to the expected direction would result in the same action as a non-significant result (a free gift would not be given away with the album), therefore you could use a one-tailed test to evaluate this hypothesis.

The confidence interval for the free-gift group ranges from 52 to 72 so has a length of 20 and an MOE of half this value (i.e., 10). For the no-free-gift group, it ranges from 50 to 56, which is a distance of 6 and an MOE of 3. The average MOE is, therefore (10 + 3)/2 = 6.5. Moderate overlap, which would indicate a significant difference between groups at p = 0.05, would be half of this value (i.e., 3.25). We can calculate the size of the overlap of the confidence intervals by subtracting the lower limit of the confidence interval for the free-gift group (52) from the upper limit of the confidence interval for the no-free-gift group (56), which gives us 4. This value is slightly greater than 3.25, suggesting that the free-gift and no-free-gift groups are unlikely to be significantly different using a criterion of p = 0.05.

We can calculate the experimentwise or familywise error rate using the following equation from the book:

In other words, when conducting 15 tests there is a 54% chance of at least one Type I error, compared to 5% when only one test is conducted.

We can correct for this error rate using the Bonferroni correction. It is not the only method , but it illustrates the general principle well. All it means is that because 15 tests were conducted, rather than accept a result as significant if its probability is below 0.05, we accept it as significant if its probability is below 0.05/15 = 0.003. In doing so, we ensure that the familywise Type I error remains below 0.05. In other words, the p-value that you would use as a threshold for significance to correct for the current error rate is 0.003.

The power of a statistical test is the probability that it will detect an effect when one actually exists.