Political Science Research Methods
Chapter Summary
This chapter discusses two related methods of collecting data from people: (1) survey research, which involves collecting information via a questionnaire or survey instrument (a carefully structured or scripted set of questions that may be administered face-to-face, by telephone, by mail, by Internet, or by other means) and (2) interviewing, which involves direct and personal communication with individuals in a less formal and less structured situation—more in the nature of a constrained conversation.
When designing a survey, researchers must consider five important factors that will guide the choice of survey type:
- Cost: All surveys involve some basic expenses for materials but the cost of a survey can be quite variable based on the survey design.
- Completion rates: If the response rate on a survey is low the likelihood that the sample is not representative of the population increases, hence the ability to make statistical inferences about the target population may be limited.
- Sample-population congruence: A particularly important problem related to completion rates is that if those who fail to participate in the survey share common characteristics, the sample will be wrought with systematic sampling error and the conclusion made with the data could be incorrect.
- Questionnaire length: A lengthy questionnaire can lead respondents to lose focus or speed through questions in order to finish sooner, resulting in low response quality. Different survey methods can use different length surveys with the maximum number of questions ranging between three or four dozen depending on complexity and format.
- Data processing issues: While advancements in personal computing have made the collection, preparation, and analysis of survey data easier, it still requires a good deal of time and effort to process data. Surveys employing more elaborate designs, a large number of questions, or many open-ended questions, will require more time and effort.
Response quality is the extent to which responses provide accurate and complete information. Quality can be better when the respondent is motivated by a friendly, businesslike survey administration style, when interviewers avoid biasing responses through cues, and when probing is used to elicit better answers.
Good questions prompt accurate answers, while poor questions prompt unreliable or inaccurate answers. In turn, unreliable or inaccurate answers lead to inaccurate results and conclusion. Surveys should avoid using double-barreled questions that ask two or more questions in one, ambiguous terms that might be misinterpreted or misunderstood, leading questions that might bias answers, and inappropriate words including technical words, slang, or unusual vocabulary.
Different question types (open or closed-ended) have advantages and disadvantages. Closed-ended questions are easy to code and analyze, but limit answers to those predetermined by the researcher. Open-ended questions allow respondents to more fully explain answers and create the opportunity for researchers to find answer choices they had not anticipated, but are more difficult to code and analyze.
The physical design of a survey can affect response quality and cause bias through the question-order effect in which previous questions affect the answers to subsequent questions, or affect the likelihood of completing the survey.
Response distributions may be affected by whether the researcher asks a single-sided question, in which the respondent is asked to agree or disagree with a single substantive statement, or a two-sided question, which offers the respondent two substantive choices.
Good questionnaire design increases the likelihood that the questionnaire will be completed properly.
Interviewing often involves a face-to-face interaction but may also take place through other direct communication like over the phone. Interviews may consist of a predetermined set of questions, a spontaneous conversation, or both. Interview material should be recorded during the interview or as soon as possible thereafter to capture as much relevant data as possible.