Classification consistency and results reporting of a digital-first computer-adaptive language proficiency test

UNCG Author/Contributor (non-UNCG co-authors, if there are any, appear on document)
Ramsey Lee Cardwell (Creator)
The University of North Carolina at Greensboro (UNCG )
Web Site:
Micheline Chalhoub-Deville

Abstract: The emergence of digital-first assessments is prompting reconsideration of, and innovation in, aspects of psychometrics, test validation, and test use. Using the Duolingo English Test (DET) as an example, this three-paper series seeks to address issues concerning the estimation of classification consistency and the reporting of results for such assessments. The first paper presents a simulation study investigating the use of CTT–based classification consistency methods in a computer-adaptive testing context. The second paper further investigates CTT–based classification consistency estimates by applying the methods from the first paper, as well as a bootstrapping-inspired approach, to operational test data from the DET. The third paper investigates the reporting of test-related information and test-taker results to results users through a focus group with North American postsecondary admissions professionals. Collectively, the studies address challenges in constructing validity arguments for digital-first high-stakes assessments.

Additional Information

Language: English
Date: 2022
Admissions testing, Classification consistency, Computer-adaptive testing, Language testing, Reliability, Results reporting
Computer adaptive testing
Educational tests and measurements
English language $x Ability testing

Email this document to