Avoiding Question Errors

Question Errors

• Leading questions suggest or influence a particular response
• "Wouldn't you agree that climate change is the most urgent issue today?" is an example of a leading question
• "How important do you think climate change is compared to other global issues?" is a better question
• Double-barreled questions combine two or more ideas into one question, making it hard to answer
• "Do you think teachers should give more assignments and extend deadlines?" is a double-barreled question
• "Do you think teachers should give more assignments?" is a better alternative
• Ambiguous or vague questions use unclear wording that may be interpreted differently by respondents
• "Do you eat healthy food?" is an ambiguous question because it's unclear what qualifies as "healthy"
• "How often do you eat fruits and vegetables in a week?" is more specific
• Loaded questions assume something that may not be true for the respondent
• "Why do you hate online classes?" assumes the respondent dislikes online classes

More Question Errors

• Using jargon involves overly complex or technical wording that respondents may not understand
• For example, "To what extent do you believe that asynchronous learning platforms enhance cognitive engagement in students?" is jargon
• "Do you think online learning platforms help students stay engaged? Why or why not?" is more clear
• Absolute questions use extreme words like always, never, or all, limiting response options
• For example, "Do you always submit your assignments on time?" is an absolute question
• "How often do you submit your assignments on time?" provides more frequency
• Double negative questions use two negatives that cancel each other out, making it unclear how to answer
• "Do you not agree that genetic engineering is not harmful?" is a double negative
• "Do you think genetic engineering is harmful?" is more direct
• Unbalanced response questions provide answer choices that are not evenly distributed
• For example, a satisfaction survey with scales of "Very satisfied, Satisfied, Neutral, Slightly satisfied" doesn't include dissatisfaction
• Having scales of "Very satisfied, Satisfied" provides balance

Instrument Design

• Contains varying types of questions
• Can be personal, attitudinal, or behavioral and also have sensitive questions where applicable
• Have strategies for good construction
• Have clear language, and answer options that don't overlap
• Ensure questions are applicable to participants
• Be valid and reliable
• Must undergo pilot testing on a smaller group of people

Coding

• Coding is the process of creating and assigning codes in order to categorize data and reduce the data, allowing themes to emerge

Quantitative Coding

• Process of categorizing collected non-numerical information into groups and assigning numerical codes
• Numeric coding is shared by all statistical software and it facilitates data conversion and measurement comparisons

Data Types and Formatting

• Quantitative Data: Requires quantifying the data, converting from the collection format into a numeric database
• Qualitative Data: Requires data processing like typing, entering, describing, and converting audio/video to text
• Combination: Involves processing each element as appropriate

Level of Measurement

• Determines the statistics
• Nominal and Ordinal involve descriptive statistics such as central tendencies, and variability
• Interval and Ratio involve inferential statistics like T-test, Pearson's r, and Regression

Reliability

• Establishing a Clear Coding Framework is important when coding
• Use Multiple Coders for intercoder reliability
• Conduct Pilot Coding
• Maintain Reflexivity & Documentation
• Use Software for Systematic Coding
• Conduct a Reliability Check
• Seek External Review

Codebook Construction

• Serves as the primary guide used in the coding process
• Notes the value assigned to each variable attribute (response)
• Guides for locating variables and interpreting codes in the data file during analysis

Types of Variable Analysis

• Univariate: Summarizes a single variable at a time; does not deal with causes, relationships, or dependent variables; the main purpose is to describe such as height
• Bivariate: Summarizes two variables; deals with causes and relationships where analysis is done; contains only one dependent variable with the main purpose of explaining temperature and ice sales in the summer
• Multivariate: Summarizes more than 2 variables, but does not deal with causes and relationships where analysis is done; similar to bivariate, but has more than 2 variables where the main propose is to study

Statistics Types

• Descriptive Statistics describe, show, summarize, and organize data, and help describe patterns, but do not draw conclusions beyond the data