These lively and stimulating ideas for use in and out of class reinforce active learning. The activities apply to individual or group projects.

1. With two other students from class, have one be “observed” while you and the other student function as independent raters (i.e., observers). Ask the person observed to periodically “look puzzled.” First, you and the second student must create your operational definition of “looking puzzled.” Then, over 10 minutes, every minute, marked when the second hand on a clock reaches 12, you and the other student independently record whether you think the person observed is looking puzzled. After you have these 10 observations, calculate Cohen’s kappa and percent agreement for the behavior. If your inter-rater reliability falls below the accepted levels [0.75 for percent agreement and 0.61 for Cohen’s kappa, using Graham et al.’s (2012) standards], meet to discuss why you think you disagreed.
    
2. Often newspaper articles provide the opportunity for readers to offer comments.
   With another class member, select a recent newspaper article from your local paper (online) that has at least 30 readers’ comments. Then, using the steps in content analysis from this chapter and working together, create the categories (with their operational definitions) into which you then place each of these comments. If you want more practice in calculating inter-rater reliability, you could each place the comments into the categories separately, and then compare your degree of agreement.