A comparison of traditional test blueprinting and item development to assessment engineering in a licensure context
- UNCG Author/Contributor (non-UNCG co-authors, if there are any, appear on document)
- James S. Masters (Creator)
- Institution
- The University of North Carolina at Greensboro (UNCG )
- Web Site: http://library.uncg.edu/
- Advisor
- Richard Luecht
Abstract: With the need for larger and larger banks of items to support adaptive
testing and to meet security concerns, large-scale item generation is a
requirement for many certification and licensure programs. As part of the mass
production of items, it is critical that the difficulty and the discrimination of the
items be known without the need for pretesting. One approach to solving this
need is item templating, an assessment engineering (AE) approach that is
intended to control item difficulty and other psychometric operating
characteristics for a class of items developed from each template. There are
important advantages that can accrue to having exchangeable items that operate
in a psychometrically similar manner in terms of item bank development (reduced
time and lower cost to develop), pretesting efficiency, test security, and so forth.
This study describes one method to use AE and item templates in a
licensure context to yield sets of items with statistical characteristics that match
the needs of the program with reduced need for pilot testing. It is shown that item
variants developed in this method fit the Rasch calibration/scoring model as well,
if not better than items developed in traditional ways and that the item variants
from the same template yield similar classical and IRT statistics. One key result
of the study is a method to use AE to evaluate the performance of item writers
over time.
A comparison of traditional test blueprinting and item development to assessment engineering in a licensure context
PDF (Portable Document Format)
524 KB
Created on 5/1/2010
Views: 3741
Additional Information
- Publication
- Dissertation
- Language: English
- Date: 2010
- Keywords
- Assessment Engineering, Item Templates, Licensure Testing
- Subjects
- Psychological tests $x Design.
- Educational tests and measurements $z United States.
- Vocational qualifications $z United States $x Measurement.
- Item response theory.
- Psychometrics.