Surveys in Research Methods

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What factor contributes to the cost-effectiveness of web-based surveys compared to custom-developed ones?

Higher response rates

No need for data entry (correct)

Lower initial setup costs

Enhanced respondent engagement

How do respondents generally feel about providing sensitive information in electronic surveys compared to face-to-face interactions?

Less honest due to anonymity concerns

More dishonest when the topic is personal

More honest when it can be submitted anonymously (correct)

Equally honest regardless of format

What is a common misconception about the honesty of respondents in online surveys compared to paper surveys?

People are more likely to lie in online surveys. (correct)

Online surveys deter individuals from participating honestly.

There is no significant difference in honesty between both formats.

Responses in paper surveys are always more trustworthy.

What is a potential advantage of web-based surveys over e-mailed surveys?

Greater anonymity for respondents Signup and view all the answers

What can reduce data entry errors in surveys?

Automatic saving of responses Signup and view all the answers

In what situation are respondents likely to be more honest in their answers?

When surveys can be submitted anonymously Signup and view all the answers

What is a challenge related to informed consent in online surveys?

Respondents are less aware of their participation. Signup and view all the answers

Which factor affects the speed of analysis in web-based surveys?

Automatic response saving Signup and view all the answers

What type of error occurs when survey questions are poorly worded or biased?

Measurement error Signup and view all the answers

Which error is exemplified by a situation where the sampling frame is split evenly by gender, but the majority of responses come from one gender?

Nonresponse error Signup and view all the answers

What can lead to coverage error in survey sampling?

Surveying individuals without a comprehensive list Signup and view all the answers

What is an example of an ordered response?

Ranking software applications from favorite to least favorite Signup and view all the answers

In which context is nonprobabilistic sampling often employed?

When it's difficult to identify a central population Signup and view all the answers

Which type of question allows respondents to choose only one item?

Closed-ended questions with ordered response Signup and view all the answers

What distinguishes unordered closed-ended questions from ordered ones?

Choices have no logical or numeric order. Signup and view all the answers

Which method is NOT typically associated with nonprobabilistic sampling?

Random digit dialing Signup and view all the answers

What is a common example of a Likert scale?

A rating from 1 to 10 of overall satisfaction Signup and view all the answers

What is the primary reason that HCI researchers often use nonprobability sampling?

The necessity to collect data individually Signup and view all the answers

Which best describes the impact of nonresponse error on research findings?

It can lead to invalid conclusions if the missing data is systematically different. Signup and view all the answers

What is a primary challenge in obtaining a random sample in certain populations, such as individuals with HIV?

Difficulty identifying and accessing potential respondents Signup and view all the answers

Which input method allows for multiple selections in a closed-ended question?

Choosing favorite social media platforms Signup and view all the answers

Which of the following statements best describes the nature of survey data in HCI research?

Surveys can be combined with other research methods for validity Signup and view all the answers

What is the primary goal of probabilistic sampling in research?

Achieving accurate estimates of the population Signup and view all the answers

In the context of survey design, which phrase best describes a figure displaying ordered responses?

A graphical representation of a Likert scale Signup and view all the answers

What distinguishes the research methodologies between social science and HCI fields?

HCI researchers collect data on their own without strict random sampling Signup and view all the answers

What type of sampling methodologies are exemplified by the National Centre for Social Research in the UK?

Rigorous, structured sampling methodologies Signup and view all the answers

What challenge might arise when creating web-based surveys with unordered response options?

Different interface widgets must be implemented. Signup and view all the answers

What is NOT a characteristic of ordered closed-ended questions?

They allow for multiple selection of answers. Signup and view all the answers

In what context are surveys commonly used by HCI researchers?

In conjunction with other research methods without expecting representativeness Signup and view all the answers

What is a common misconception about sampling in HCI research?

Only probability-sampled data can yield valid results Signup and view all the answers

How are survey responses treated in recent HCI research papers, according to the discussed practices?

Surveys are complemented by various qualitative methods Signup and view all the answers

What is a notable feature of survey research within the HCI community?

There is acceptance of convenience samples Signup and view all the answers

What is a key factor in increasing the response rate of a survey?

Establishing credibility through trusted authorities Signup and view all the answers

Which step is NOT part of the suggested multistep contact process for increasing survey response rates?

Conducting face-to-face interviews Signup and view all the answers

Why is it important to identify the source of the survey?

To establish authority and credibility Signup and view all the answers

What is one method to simplify the return of a paper survey?

Providing a self-addressed return envelope with postage Signup and view all the answers

What should be considered when determining the number of survey responses needed?

The goal of the survey and the sampling method Signup and view all the answers

What is the purpose of a final contact using a different mode?

To increase the likelihood of response through novelty Signup and view all the answers

What is the main idea behind multiple reminders in electronic surveys?

To increase the probability of a response Signup and view all the answers

Which method is NOT recommended for making contacts with nonrespondents?

Sending the final survey through the same postal method Signup and view all the answers

What is the primary purpose of conducting a pilot study for a survey tool?

To ensure the questions are clear and unambiguous Signup and view all the answers

Which of the following is NOT a stage in the three-stage process of pretesting a survey?

Analysis of survey marketing effectiveness Signup and view all the answers

In the context of pilot testing, what role do knowledgeable colleagues play?

They evaluate the clarity of the survey tool before testing with users Signup and view all the answers

What aspect of a survey tool's interface can significantly affect its usability?

The font face, size, and spacing used on paper surveys Signup and view all the answers

What often happens in practice, even though a three-stage pilot testing process is ideal?

Minimal level of pilot testing is performed with limited colleague input Signup and view all the answers

What is the ultimate goal of the pilot study once potential respondents complete the survey?

To identify confusing or misleading questions Signup and view all the answers

When evaluating paper-based surveys, which layout aspect is crucial?

Use of grids and spacing Signup and view all the answers

What quality of the survey tool is assessed through interviews with potential respondents?

Clarity and motivational qualities of the questions Signup and view all the answers

Study Notes

Surveys

Surveys are a common research method used across various fields, including human-computer interaction (HCI).
They are used to describe populations, explain behaviors, and explore new areas.
Surveys can be structured, well-tested, and robust, resulting in valid data. However, poor design can lead to unreliable data.
A survey is a set of well-defined questions that individuals answer independently.
Surveys collect data quickly from large, geographically diverse populations.
Random sampling in surveys allows for statistically accurate population estimates.
Surveys are sometimes criticized for being too simple a method, used inappropriately for generalizations.

Benefits and Drawbacks of Surveys

Surveys are cost-effective for gathering data from a large number of people quickly.
Surveys are relatively unobtrusive compared to other methods.
They provide a broad overview of user interactions, problems, and actions.
Surveys are not suitable for in-depth analysis, gathering detailed information, or for testing specific usability issues within an interface.
Poorly designed surveys can produce unreliable data that does not generalize to other populations.
Survey questions related to user opinions or habits may be more prone to recall bias compared to factual questions.

Combining surveys and computer-collected data allows for a more holistic understanding of user behavior.
Surveys can provide a general overview of behavior/attitude and motivations.
A survey on photo tagging behavior, for instance, allowed for more accurate data collection.
Random sampling methods allow for more representative samples of user populations.

Goals and Targeted Users for Survey Research

Surveys can be used for understanding large populations.
The targeted population of interest needs to be well-defined.
Clear criteria must be used in determining target population to effectively use surveys.
The demographics of the targeted users are important, for analyzing validity, representing, and ensuring a diverse population.
Identifying individuals or groups with particular characteristics (age, profession, disabilities etc.) is crucial.

Probabilistic Sampling

A census is the most accurate method for estimating a whole population, but it's expensive
Probabilistic sampling is used when conducting surveys, ensuring each participant has a specific likelihood of being selected.

Stratification

Stratified sampling involves dividing the target population into subgroups and taking a sample from each.
This approach is useful when subpopulations have unequal representation.
It ensures more representative subgroups within the sample.

Response Size

The sufficient response size depends on desired confidence level, margin of error.
A larger sample size reduces margin of error, increasing accuracy.

Nonprobabilistic Sampling

When a complete list of all target population members isn't available, nonprobabilistic approaches are preferred.
Nonprobabilistic samples include volunteer opt-in panels and self-selected surveys.
Demographic data and oversampling might be employed to improve validity in nonprobabilistic surveys.

Developing Survey Questions

Open-ended questions offer respondents flexibility but require more complex analysis.
Closed-ended questions provide quantifiable data, but require carefully crafted, unbiased options.
Common survey errors to avoid include double-barreled questions, negatively worded questions, and loaded phrasing.

Overall Survey Structure

Surveys should begin with clear instructions/guidelines for completing the survey.
Order of questions may be critical; for instance, sensitive questions are placed later.
White space, clear formatting, navigation, contact details, and consistent layouts promote clarity for respondents.

Pilot Testing the Survey Tool

Pilot testing helps in ensuring survey questions are clear/unambiguous and questions provide expected type of data/responses.
Testing should involve expert users/analysts and potential respondents, to evaluate any flaws/confusions/misleading questions immediately.

Existing Surveys

For efficient survey development, researchers can utilize existing validated HCI survey tools.
They can modify existing tools to fit research needs.

Paper or Online Surveys?

Paper or online surveys depend heavily on the access to the target sample.
Accessibility of tools and survey formats are crucial to participation of diverse audiences, including people with disabilities.
A combination of paper and online approaches might be suitable to potentially reach a broader range of participants.

Data Analysis

Survey data analysis methods vary based on the type of questions used (closed-ended or open-ended).
Quantitative data analysis focuses on descriptive statistics, which summarize data.
Qualitative data analysis involves deeper interpretation, including coding and thematic analysis, of open-ended responses.

Summary

Surveys are powerful tools but their use requires careful consideration of design, methodology, and intended implications.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

Explore the role of surveys as a key research method in various fields, including human-computer interaction. This quiz delves into the benefits and drawbacks of surveys, their design, and their impact on data reliability. Test your knowledge on how surveys gather data efficiently and their limitations.