Session Review: Assessment of knowledge: Test Items that Measure Learning Effectively

(May 3)

Summary: In this session, Atena presented the process and procedure that the TD Bank Financial Group uses to develop their test items. The strategy was to develop performance-based items that measure on-the-job transferable knowledge and skills. Through the analysis process, they determine if testing is required and, if yes, ensure that the test items map to the learning objectives, which map to the performance objectives, which map to the business objectives. Performance-based items focus more on testing conceptual, process, procedure, and principle-based content which usually requires an element of judgment or decision-making. This is in contrast to test items that focus on facts and access recall rather than higher-level cognitive skills. Also covered are item writing strategies, establishing validity and reliability, and testing the test.

Ang’s thoughts: A new approach that I came away with is the value of asking some L3 questions on the L1 evaluation to set a baseline. This session was more a confirmation of my existing knowledge of test item development and processes – always a nice thing. One thing that Atena reinforced was that the writing of test items is not an easy task. Perhaps the development of the stem is the easiest piece, but coming up with feasible distractors is a real tough one. It was mentioned in an audience discussion that the expected number of good test items that someone would write on a day is somewhere between 8 and 10. This may surprise most people, but perhaps not those who have actually been involved in creating good test items.

Submitted by: Angela van Barneveld