Survey Methods & Design - lec 4

• The lecture is on survey methods, measurement, and questionnaires.
• Next week's class will be online and focused on pilot surveys.
• The Zoom link will be available on Quercus.
• A midterm exam will occur in two weeks during class.
• There are meeting dates in February to provide feedback on the current proposals.
• These are 15-20 minute meetings.
• It is possible to email to schedule 3 or more hour-long time slots.

Survey Methods

• Surveys involve interviewing many respondents using a predesigned questionnaire.
• Person-administered surveys involve an interviewer reading questions and recording answers face-to-face or over the phone.
• Computer-administered surveys use technology to pose questions and record responses.
• Popular online survey sites are Qualtrics, Survey Monkey, Google Forms, and MS Forms.
• Self-administered surveys involve respondents completing the survey on their own
• Mixed-mode surveys use a combination of methods.

Person-Administered Surveys

• Advantages include feedback/adaptability, rapport, quality control, and relatively good response rates.
• Disadvantages include human error proneness, control of "interviewer effects" that may depend on interviewer.
• Other disadvantages include high cost and "interview evaluation".
• "Interview Evaluation" is when subjects want to give the answer they believe is "correct" to the interviewer, which is a problem with sensitive topics.

Computer-Administered Surveys

• Advantages include a breadth of user-friendly features and a quick, inexpensive study.
• These also reduce interview evaluation effects and have automatic data input/collection.
• Disadvantages include the need for computer-literate and/or internet-connected respondents, and potential cooperation issues.

Self-Administered Surveys

• Advantages are reduced cost, respondent control over pacing, and no interview-evaluation apprehension.
• Disadvantages are that nonresponse can exacerbate self-selection bias, with a lack of monitoring, and high questionnaire requirements.
• Nonresponse can exacerbate self-selection bias with systematic differences between those who respond and those who don't, making the sample unrepresentative.

Mixed-Mode Surveys

• Advantages include multiple benefits to achieve data collection goals.
• A benefit is the potential to use online surveys to quickly reach part of the population.
• It is also possible to reach those unreachable via online via telephone.
• These can also alleviate problems associated with any one method.
• Disadvantages include the possibility that "mode" will affect the response, and additional complexity.

Mode May Affect Response

• Studies demonstrate phone and online surveys can yield different results.
• Online survey responses may have more "negativity" than those in phone surveys.
• Individuals surveyed over the phone are more likely to state they're "very satisfied" with family and social life, and less likely to express "very unfavorable" views of politicians.

Measurement

• Measurement involves determining whether or how much of a property is possessed by an object.
• Objective properties are physically verifiable like age, height, or bottles purchased.
• Subjective properties are mental constructs not directly observable, such as attitudes, opinions, or intentions.
• Subjective properties are usually translated onto an "intensity continuum".
• Scale development designs questions and response formats to measure subjective properties.

Types of Measures

• There are 4 types of measures: nominal, ordinal, scale, ratio and interval.
• Nominal measures use only labels.
• Ordinal measures are such that responses can be ranked.
• Scale responses mean that the distance between levels is known.
• Ratio responses involve when a true zero exists.
• Interval responses mean that there are equal distances between levels (implicit).

Examples of the Types of Measures

• How many years old are you would be Nominal.
• Are you left-handed, right-handed, or ambidextrous would be Ordinal.
• Ranking apples, oranges, and pears from most to least favorite would be Interval.
• Rating customer service on a 1-10 scale with 1 being the worst and 10 being the best would be Ratio.

Smartphone Brand Preferences

• Choosing which phone brand you like most is Nominal.
• Raking four phone brands from 1 to 4, with 1 being most preferred and 4 being least preferred is Ordinal.
• Indicating how much you like each brand from 1 to 5, where 1 is "don't like it at all" and 5 is "like it a lot" is Interval.
• Indicating how likely you'd be to choose each brand if you were to buy a new phone, in percent, is Ratio.

Likert Scale

• With Likert scales, respondents indicate their level of agreement or disagreement on a symmetric agree-disagree scale for each statement.
• This is commonly used for lifestyle inventories.
• Lifestyle inventories segment the market based on personality traits, interests, values, and lifestyles.

Semantic Differential Scales

• Semantic differential scales feature a sequence of bipolar adjectives for an object's properties, so respondents indicate their impressions of each property on a continuum.
• Halo effects occur when an overall like or dislike of the object can bias responses on its specific properties.

Questionnaire Design

• Questionnaire design involves the question format, wording, and layout of questionnaire.
• Questionnaire organization refers to the sequence of questions.
• Poorly organized questionnaires discourage and may cause people to stop answering.
• Questionnaire development selects response formats while wording to be understandable, unambiguous, and unbiased.
• Minimized question bias is important to the design.

Types of Questions

• Types of questions include warm-up, classification/demographic, screening, transitions, and "tough" questions.
• Warm-up questions are simple, easy-to-answer questions used to get respondents interested to complete the rest of the survey.
• Usually these can also provide information relevant to the objective using information like social media platforms, etc.
• Warm-ups in person-administered surveys are verbalized and open-ended.
• Classification/demographic questions classify to aid analysis using questions about age, gender, income, etc.
• Screening questions "screen out" respondents who are not eligible with "eligibility" defined depending on the research objectives.
• Transition questions inform the respondent of upcoming topic of format changes.
• "Tough" questions are more complicated or harder to answer however they are relevant and address the objectives of research.
• Tough questions would be crucial in an easier question.

Ordering of Questions

• Warm-up questions should be asked first.
• Classification/demographic questions should be right after warm-up questions.
• Screening questions should be the start of a new section featuring a new topic or question format.
• Transition questions should be asked in the middle or towards the end of the survey.
• "Tough" questions should be asked last.

Test your knowledge of survey methodologies, including administration types, potential biases, and questionnaire design. Questions cover advantages of computer surveys, interviewer effects, response rates, and the impact of survey topics.