Applying the Pseudo-Panel Approach to International Large-Scale Assessments

Abstract

TIMSS and PIRLS assess representative samples of students at regular intervals, measuring trends in student achievement and student contexts for learning. Because individual students are not tracked over time, analysis of international large-scale assessment data is usually conducted cross-sectionally. Gustafsson (2007) proposed examining the data longitudinally by analyzing relationships between country-level trends in background constructs and trends in student achievement. Through longitudinal analysis of international large-scale assessment data, it becomes possible to mitigate some of the confounding factors in the analysis. This dissertation extends this country-level approach to subpopulations within countries. Adapting a pseudo-panel approach from the econometrics literature (Deaton, 1985), the proposed approach creates subpopulations by grouping students based on demographic characteristics, such as gender or parental education. Following grouping, the subpopulations with the same demographic characteristics are linked across cycles and the aggregated subpopulation means are treated as panel data and analyzed through longitudinal data analysis techniques. As demonstrated herein the primary advantages of the subpopulation approach are that it allows for analysis of subgroup differences, and it captures within-country relationships in the data that are not possible to analyze at country level. Illustrative analysis examines the relationship between early literacy activities and PIRLS reading achievement using PIRLS 2001 and PIRLS 2011 data. Results from the subpopulation approach are compared with student-level and country-level cross-sectional results as well as country-level longitudinal results. In addition, within-country analysis examines the subpopulation-level relationship between early literacy activities and PIRLS reading achievement, multiple group analysis compares regression coefficient estimates between boys and girls and across parental education subgroups, and mediation analysis examines the extent that partaking in early literacy activities can explain differences between boys and girls in PIRLS reading achievement.