Exploring Self-Reported Survey Data in Higher Education as an Artifact of Socio-Environmentally Influenced Behavior

Abstract

Much of the research about college student engagement is based upon self-reported surveys, but little is known about how students formulate responses to these instruments. The purpose of this study was to specifically address this dearth of knowledge by deepening our understanding of how students’ perceptions of their environments and demographic characteristics influenced their response patterns on self-reported surveys. Bronfenbrenner’s (2005) human ecology model of development, Bourdieu and Passeron’s (1990) theory of social reproduction, and Tourangeau, Rips, and Rasinski’s (2000) four phase survey response process were used, as the theoretical framework to better understand this phenomenon. This was an explanatory sequential mixed methods study, and the participants were first-year undergraduate students at a four-year, private institution in New England. Students completed the College Student Report (CSR) as well as a series of time-use diaries, and the results of the instruments were compared using descriptive and multivariate analyses. Finally, semi-structured individual interviews were conducted, which included aspects of retrospective cognitive interviewing, with twenty-seven (27) students to understand how their experiences and response processes were shaped by their individual campus experiences and identities. Findings from this study suggest the construct validity of self-reported survey data measuring behavioral frequency patterns is questionable, as students statistically significantly under reported time spent preparing for class, engaging in co-curricular activities, commuting to campus, and relaxing and socializing. Furthermore, student characteristics such as racial/ethnic identity and satisfaction with college choice statistically significantly explained some of the variance in the reporting behaviors of students after controlling for other factors. This information coupled with the data gleaned from the semi-structured individual interviews indicate factors related to how students differentially experience the campus environment based upon their unique ecological niches affects how they respond on self-reported surveys, which means the data provided by such instrumentation is likely providing substantively different information than how it is most commonly interpreted and applied.