Hypothesis testing

Content

Sampling variability
Hypothesis testing
Pairwise comparisons

Learning objectives

At the end of the session, students should be capable of

understanding the mechanics behind generic hypothesis tests
interpreting the output of generic tests
correctly reporting the output of a testing procedure

Readings

Chapter 5 (Foundations for inference) of Matthew Crump’s course notes. These notes are non-technical, but do a good job at explaining the notion of sampling variability and chance. If you find them too basic, skip directly to the next item.
Chapter 2 of the Course notes
The permutation test by Jared Wilson

Complementary readings

Chapter 2 of Maxwell et al. (2017)
Chapter 3 of Keppel & Wickens (2004).

Slides

View all slides in new window Download PDF of all slides

Tip

Fun fact: If you type ? (or shift + /) while going through the slides, you can see a list of slide-specific commands.

Case study

We will look at the way authors report the conclusion of their statistical tests with

Rosen & Jerdee (1974)
Brucks & Levav (2022)
Liu et al. (2023), Experiment 1

Code

The following code reproduces the applications in the course slides and the case study

References

Brucks, M. S., & Levav, J. (2022). Virtual communication curbs creative idea generation. Nature, 605(7908), 108–112. https://doi.org/10.1038/s41586-022-04643-y

Keppel, G., & Wickens, T. D. (2004). Design and analysis: A researcher’s handbook. Pearson Prentice Hall.

Liu, P. J., Rim, S., Min, L., & Min, K. E. (2023). The surprise of reaching out: Appreciated more than we think. Journal of Personality and Social Psychology, 124(4), 754–771. https://doi.org/10.1037/pspi0000402

Maxwell, S. E., Delaney, H. D., & Kelley, K. (2017). Designing experiments and analyzing data: A model comparison perspective (3rd ed.). routledge. https://doi.org/10.4324/9781315642956

Rosen, B., & Jerdee, T. H. (1974). Influence of sex role stereotypes on personnel decisions. Journal of Applied Psychology, 59, 9–14. https://doi.org/10.1037/h0035834