Detecting test cheating using a Deterministic, gated item response theory model

UNCG Author/Contributor (non-UNCG co-authors, if there are any, appear on document)
Zhan Shu (Creator)
Institution
The University of North Carolina at Greensboro (UNCG )
Web Site: http://library.uncg.edu/
Advisor
Richard Luecht

Abstract: High-stakes tests are widely used as measurement tools to make inferences about test takers' proficiency, achievement, competence or knowledge. The stakes may be directly related to test performance, such as obtaining a high-school diploma, being granted a professional license or certificate, etc. Indirect stakes may include state accountability where test results are partially included in course grades and also tied to resource allocations for schools and school districts. Whether direct or indirect, high stakes can create an incentive for test cheating, which, in turn, severely jeopardizes the accuracy and validity of the inferences being made. Testing agencies and other stakeholders therefore endeavor to prevent or at least minimize the opportunities for test cheating by including multiple, spiraled test forms, minimizing item exposure, proctoring, and a variety of other preventive methods. However, even the best test prevention methods cannot totally eliminate cheating. For example, even if exposure is minimized, there is still some chance for a highly motivated group of examinees to collaborate to gain prior access to the exposed test items. Cheating detection methods, therefore, are developed as a complement to monitor and identify test cheating, afterward. There is a fairly strong research base of statistical cheating detection methods. However, many existing statistical cheating detection methods are in applied settings. This dissertation proposes a novel statistical cheating detection model, called the Deterministic, Gated Item Response Theory Model (DGIRTM). As its name implies, the DGIRTM uses a statistical gating mechanism to decompose observed item performance as a gated mixture of a true- proficiency function and a response function due to cheating. The gating mechanism and specific choice of parameters in the model further allow estimation of a statistical cheating effect at the level of individual examinees or groups (e.g., individual suspected of collaborating). Extensive simulation research was carried out to demonstrate the DGIRTM's characteristics and power to detect cheating. These studies rather clearly show that this new model may significantly improve our capability to sensitively detect and proactively respond to instances of test cheating.

Additional Information

Publication
Dissertation
Language: English
Date: 2010
Keywords
Exposure, MCMC, Test Cheating
Subjects
Cheating (Education) $x Statistics.
Educational tests and measurements.
Monte Carlo method.
Markov processes.

Email this document to