The effects of mixture-induced local dependence on diagnostic classification

UNCG Author/Contributor (non-UNCG co-authors, if there are any, appear on document)
Thomas P. McCoy (Creator)
The University of North Carolina at Greensboro (UNCG )
Web Site:
John Willse

Abstract: Diagnostic Classification Models (DCMs) have been extensively researched in recent psychometric literature for providing mastery skill profiles for diagnostic feedback (Henson, Templin, & Willse, 2009). DCMs are multidimensional confirmatory latent class models (LCMs) where latent classes represent skill mastery profiles and latent attributes are categorical (mastery or non-mastery). DCMs make a central assumption that once mastery profiles are accounted for that items are independent, referred to as local independence (LI). Construct irrelevant variance (e.g., differential item functioning (DIF), speededness, test wiseness, item-to-skill misspecification) or underrepresentation (extra dimensionality, inappropriate definitional grain-size of defined skills) could introduce systematic within-class variation which would violate LI. Using connections of LCMs with mixture IRT models, this study explores the effects of introducing systematic within-class variation on diagnostic classification. The log-linear cognitive diagnosis model (LCDM) is extended to include continuous abilities, akin to a multidimensional item response theory (MIRT) model with underling mixtures due to skill mastery/nonmastery. Data were then simulated for different ability variances related to distribution overlap conditions. Multiple DCMs are then fit using the LCDM framework in a simulation study. Impact on classification and local dependence detection are summarized. It was found that as mixture overlap increased due to companion ability variance that diagnostic classification in DCMs greatly suffered, but can be detected by Yen’s Q3. The relationship of the degree of inaccuracy and effect sizes based on ability variance and group separation is delineated. Recommendations for practitioners are given along with areas for future study.

Additional Information

Language: English
Date: 2015
Classification, Diagnostic models, Latent classes, Multidimensional, Psychometrics, Simulation study
Educational tests and measurements
Examinations $x Evaluation

Email this document to