A comparison of traditional test blueprinting and item development to assessment engineering in a licensure context

UNCG Author/Contributor (non-UNCG co-authors, if there are any, appear on document)
James S. Masters (Creator)
Institution
The University of North Carolina at Greensboro (UNCG )
Web Site: http://library.uncg.edu/
Advisor
Richard Luecht

Abstract: With the need for larger and larger banks of items to support adaptive testing and to meet security concerns, large-scale item generation is a requirement for many certification and licensure programs. As part of the mass production of items, it is critical that the difficulty and the discrimination of the items be known without the need for pretesting. One approach to solving this need is item templating, an assessment engineering (AE) approach that is intended to control item difficulty and other psychometric operating characteristics for a class of items developed from each template. There are important advantages that can accrue to having exchangeable items that operate in a psychometrically similar manner in terms of item bank development (reduced time and lower cost to develop), pretesting efficiency, test security, and so forth.

This study describes one method to use AE and item templates in a licensure context to yield sets of items with statistical characteristics that match the needs of the program with reduced need for pilot testing. It is shown that item variants developed in this method fit the Rasch calibration/scoring model as well, if not better than items developed in traditional ways and that the item variants from the same template yield similar classical and IRT statistics. One key result of the study is a method to use AE to evaluate the performance of item writers over time.

Additional Information

Publication
Dissertation
Language: English
Date: 2010
Keywords
Assessment Engineering, Item Templates, Licensure Testing
Subjects
Psychological tests $x Design.
Educational tests and measurements $z United States.
Vocational qualifications $z United States $x Measurement.
Item response theory.
Psychometrics.

Email this document to