Examining the Impact of Accommodations and Universal Design on Test Accessibility and Validity

Abstract

Large-scale assessments are often used for statewide accountability and for instructional and institutional planning. It is essential that the instruments used are valid and reliable for all test takers included in the testing population. However, these tests have often fallen short in the area of accessibility, which can impact validity for students with special needs. This dissertation examines two strategies to addressing accessibility: the use of technology to implement principles of universal design to assessment and the provision of accommodations. This study analyzed test data for students attending high schools in New Hampshire, Vermont and Rhode Island who participated in the 2009 11th grade New England Common Assessment Program (NECAP) science assessment. Three test conditions were of interest: (1) no accommodations with a paper-based form (2) accommodated test administration with a paper-based form and (3) accommodated test administration using a universally designed computer-based test delivery system with embedded accommodations and accessibility features. Results from two analyses are presented: differential item functioning (DIF) and confirmatory factor analysis (CFA). DIF was used to explore item functioning, comparing item difficulty and discrimination under accommodated and non-accommodated conditions. Similarly, CFA was used to examine the consistency of underlying factor structure as evidence that constructs measured were stable across test conditions. Results from this study offered evidence that overall item functioning and underlying factor structure was consistent across accommodated and unaccommodated conditions, regardless of whether accommodations were provided with a paper form or a universally designed computer-based test delivery system. These results support the viability of using technology-based assessments as a valid means of assessing students and offering embedded, standardized supports to address access needs.