Learning Objectives

After completing your study of this chapter, you should be able to do the following:

• Describe how to define the test domain, the target audience, and the purpose of the test.
• Develop a test plan, including defining the construct or constructs the test measures, choosing the test format, and specifying how to score the test.
• Differentiate between objective and subjective test questions, and describe the strengths and weaknesses of each.
• Describe the important issues in writing instructions for the test administrator and the test taker.

Chapter Summary

This chapter has discussed the initial steps for developing psychological tests. The first step in test devel­opment is defining the testing universe, the target audience, and the purpose of the test. This stage pro­vides the foundation for all other development activities. After a review of the literature, the test devel­oper writes a concise definition of the construct, operationalizing the construct in terms of observable and measurable behaviors. The next step is to write a test plan that specifies the characteristics of the test, including a definition of the construct, the content to be measured (the testing universe), the format for the questions, and instructions for administering and scoring.

The cumulative model of scoring—which assumes that the more the test taker responds in a particu­lar fashion, the more the test taker exhibits the attribute being measured—is probably the most common method for determining an individual’s final test score. The categorical model and the ipsative model are other scoring methods.

After completing the test plan, the test developer begins writing the test questions and instruc­tions. The questions, which test developers call items, can be in an objective format (multiple choice, true/false, and forced choice) or a subjective format (essays and interviews). Projective tests are another type of subjective format that uses ambiguous stimuli (words or pictures) to elicit responses from the test taker.

Some people have response sets—patterns of responding that result in false or misleading infor­mation—such as social desirability, acquiescence, random responding, and faking that cause test scores to contain error. Therefore, test developers need to be aware of these types of responses and guard against them. Although there is no set of rules that guarantees that items will perform as expected, the test development literature contains a number of suggestions for writing successful items.

Objective items provide ample sampling of the testing universe, but they are more time-consuming to develop. Scoring of objective items is easier and likely to be more accurate and reliable. Subjective items are easier to construct and revise. Some experts suggest that essay and interview questions are better suited for testing higher order skills such as creativity and organization. Scoring of subjective items is more difficult, requiring independent scoring by two experts to increase reliability and accuracy. There also more complex test items such as performance assessments, simulations and portfolios. These items are subject to the same technical standards as any other test item.

Although the test items make up the bulk of the new test, they are meaningless without specific instructions on how to administer and score the test. The test developer should write three sets of instructions: one for the administrator, another for the test taker, and a third for the person who scores and interprets the test results.

After writing the test items, the test developer conducts a pilot test to determine the final form the test will take. The test developer then follows up the pilot test with other studies that provide the neces­sary data for validation and norming. Conducting the pilot test and analyzing its data are integral parts of the test development process. (We address them in detail in Chapter 12.)