Problem set 12

Complete this task individually.

Submission information: please submit on ZoneCours

Elliott et al. (2021) attempt to replicate a study of Flavell et al. (1966) and study unprompted verbalization by children aged 5 to 10 in an experiment. The data containing the counts of the number of verbalization across all labs are given in the FBC66_T1 and MULTI21_D1 datasets. You can also download the SPSS databases via these links (FBC66_T1 and MULTI21_D1). Questions 1–3 refer to the FBC66_T1 data, the remainder to the MULTI21_D1 data.

The data from Table 1 of Flavell et al. (1966) are the number of children who verbalized without prompting overall in the first two experimental conditions (immediate recall and delayed recall subtasks).1

For the \(\chi^2\) test to be valid, we need the expected number of counts to be at least five in each subcondition (otherwise the large sample approximation breaks down and is unreliable).

The number of students for each grade (kindergarden, second and fifth) is 20. The rows of the table are the ordered number of instances of verbalization.

# Reinstall package
# remotes::install_github("lbelzile/hecedsm")
data(FBC66_T1, package = "hecedsm")
chisq.test(xtabs(count ~ grade + frequency, data = FBC66_T1))

    Pearson's Chi-squared test

data:  xtabs(count ~ grade + frequency, data = FBC66_T1)
X-squared = 28.358, df = 4, p-value = 1.055e-05
  1. Looking at the output, what is the conclusion of the \(\chi^2\) test?
  2. Flavell et al. (1966) report a pairwise test for second versus fifth grade only. Reproduce their analysis by discarding the row for kindergarden students: do the results match those reported in the paper?
  3. Usual recommendations for the chi-square test are that the expected counts in each cell be at least 5. Under the null hypothesis of equal frequency per grade, what is the expected number of students in each cell? Hint: calculate the marginal proportion per column.2
  1. The replication study of Elliott et al. (2021) consider children of different ages corresponding roughly to Flavell et al. (1966), but with an additional year. Repeat the \(\chi^2\) test for the MULTI21_D1 data. Perform pairwise comparisons (Hint: in R, use xtabs to cross-classify counts of factor).
  2. Why is the \(p\)-value so small in their replication?

References

Elliott, E. M., Morey, C. C., AuBuchon, A. M., Cowan, N., Jarrold, C., Adams, E. J., Attwood, M., Bayram, B., Beeler-Duden, S., Blakstvedt, T. Y., Büttner, G., Castelain, T., Cave, S., Crepaldi, D., Fredriksen, E., Glass, B. A., Graves, A. J., Guitard, D., Hoehl, S., … Voracek, M. (2021). Multilab direct replication of Flavell, Beach, and Chinsky (1966): Spontaneous verbal rehearsal in a memory task as a function of age. Advances in Methods and Practices in Psychological Science, 4(2), 1–20. https://doi.org/10.1177/25152459211018187
Flavell, J. H., Beach, D. R., & Chinsky, J. M. (1966). Spontaneous verbal rehearsal in a memory task as a function of age. Child Development, 37(2), 283–299. http://proxy2.hec.ca/login?url=https://search.ebscohost.com/login.aspx?direct=true\&db=pbh\&AN=10398223\&lang=fr\&site=ehost-live

Footnotes

  1. The last task, point-and-name, is not considered here because it specifically instructed students to speak out loud.↩︎

  2. For a two-way contingency table under the null hypothesis of no interaction, the expected counts \(E_{ij}\) of cell \((i,j)\) is obtained as the (total of row \(i\)) times (total of column \(j\)), divided by the overall count. If the expected number is less than 5, the asymptotic \(\chi^2\) approximation is dubious. We could resort to permutation and simulation to assess the accuracy of the \(\chi^2\) benchmark. The row sum for the counts (per grade) is fixed to 20, but the columns are allowed to vary.↩︎