Week 4/5/6 + 9/10

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is one advantage of using the internet for research?

Limited access to specific groups

Higher costs in recruitment

Potentially less experimenter bias (correct)

Slower data collection

Which of the following statements reflects an ethical disadvantage of internet research?

An increased motivation among younger participants

Increased self-disclosure of sensitive information

Harm resulting from direct participation (correct)

Potential for diverse sample size

What issue is associated with sample biases in internet research?

Higher motivation to complete surveys

Drop out rates among participants (correct)

Standardized data collection conditions

Increased generalisability of findings

What is a common method used for online data collection?

Qualtrics Signup and view all the answers

Which of the following is NOT an advantage of using the internet for research?

Limited recruitment possibilities Signup and view all the answers

What was a concern regarding the Facebook emotional manipulation study?

Potential breaches of confidentiality Signup and view all the answers

What makes online surveys more appealing to younger participants?

Familiarity with digital platforms Signup and view all the answers

What is a limitation of internet-based research settings?

Lack of control over data collection setting Signup and view all the answers

What is one identified threat to data quality in research surveys?

Technology failures Signup and view all the answers

Which step is not part of the survey research process?

Analyze competitor surveys Signup and view all the answers

What is crucial to avoid multiple submissions in survey research?

Recording IP addresses Signup and view all the answers

What aspect of a measure relates to its ability to produce consistent results?

Internal consistency Signup and view all the answers

When creating a survey, what is a vital factor for ensuring the measure's quality?

The method of delivery Signup and view all the answers

What might be a reason to adapt an existing measure in research?

It conflicts with participant demographics Signup and view all the answers

Which method is recommended for handling invalid responses in survey data?

Establishing actual exclusion criteria Signup and view all the answers

What is a key consideration when determining whether to create a new measure?

The cost associated with the measure Signup and view all the answers

What is a significant drawback of online surveys?

They may not access populations without internet. Signup and view all the answers

Which type of data includes personal information related to sensitive topics?

Highly sensitive data Signup and view all the answers

What is one of the first steps in creating a new online survey?

Determining the sensitivity level of the data Signup and view all the answers

What is one of the primary advantages of online surveys?

They provide access to large samples anywhere. Signup and view all the answers

What is a potential issue that could arise from participant incentives in surveys?

Induction of patterned responding Signup and view all the answers

Which category does data containing ecological details about endangered species fall into?

Identifiable/re-identifiable sensitive data Signup and view all the answers

What is a recommended method for distributing a survey?

Personal or professional email Signup and view all the answers

Which type of data can be shared publicly and is aggregated?

General data Signup and view all the answers

What is a key benefit of using or adapting an existing measure in research?

It saves time and resources. Signup and view all the answers

Which scale is associated with measuring attitudes?

Strongly disagree, Disagree, Neutral, Agree, Strongly agree Signup and view all the answers

What is an important focus when drafting survey items to ensure reliability?

Ensuring items are interpreted consistently across different respondents. Signup and view all the answers

What should be done after drafting survey items?

Pilot and edit the items based on feedback. Signup and view all the answers

Which of the following is a disadvantage of using a mail survey?

Poor response rates. Signup and view all the answers

What is a challenge associated with face-to-face or phone interviews?

They can limit the sample size due to high costs. Signup and view all the answers

When creating a survey, what should be considered regarding the type of survey to use?

The population’s access and convenience should be evaluated. Signup and view all the answers

What is a potential downside of using a mail survey?

Manual data entry issues. Signup and view all the answers

What is the potential impact of participants engaging in highly inattentive or careless responding?

It can obscure meaningful results. Signup and view all the answers

Which of the following types of invalid responses refers to selecting the same answer for each question?

Straight liners Signup and view all the answers

What is one method to detect inattentive responding during a survey?

Including self-reported diligence measures Signup and view all the answers

What is one consequence of deleting cases with inappropriate responses from a dataset?

It reduces statistical power. Signup and view all the answers

How might one conduct a sensitivity analysis in response to suspected invalid responses?

Conduct analyses with and without the invalid responses. Signup and view all the answers

Which statement describes 'faking good' in survey responses?

Presenting oneself in an overly positive light. Signup and view all the answers

What does the term 'survey bots' refer to in the context of survey validity?

Programs that can automatically fill out surveys. Signup and view all the answers

What does completing a survey in a suspiciously short amount of time indicate?

Possibility of inattentive responding. Signup and view all the answers

What is a characteristic of highly sensitive data?

It includes financial details and genetic data. Signup and view all the answers

Which type of data is NOT identifiable or re-identifiable?

General data in aggregated form. Signup and view all the answers

What is a significant ethical benefit of conducting research via the internet?

Increased anonymity and self-disclosure Signup and view all the answers

What is a primary disadvantage of using online surveys?

There is an increased risk of survey fraud. Signup and view all the answers

Which of the following is a disadvantage unique to internet-based research?

Potential for sample biases Signup and view all the answers

What is a challenge posed by the lack of control over data collection settings in online research?

Potentially less energy invested by participants Signup and view all the answers

What step is essential before creating a new online survey?

Determining the data sensitivity level. Signup and view all the answers

Which of the following best describes sensitive data?

It contains personal information not related to health or finances. Signup and view all the answers

Which factor can reduce the risk of experimenter bias in online research?

Use of internet over face-to-face methods Signup and view all the answers

What is a common concern related to participant distress in online studies?

Awareness of emotional manipulation Signup and view all the answers

What is a recommended approach to ensure data quality in survey research?

Establish objective exclusion criteria Signup and view all the answers

Which property of a measure assesses its ability to yield consistent results over time?

Reliability Signup and view all the answers

How can researchers prevent multiple submissions from the same participant in online surveys?

Record IP addresses during data collection Signup and view all the answers

When determining the quality of a measure, which aspect evaluates its ability to measure what it intends to?

Validity Signup and view all the answers

What is an important step to take after drafting survey items for a new survey?

Conduct a pilot test Signup and view all the answers

Which factor should be considered when choosing an existing measure for adaptation?

Existing psychometric properties Signup and view all the answers

Which survey distribution method is likely to reach a broader audience efficiently?

Online surveys Signup and view all the answers

Which of the following is a benefit of using an existing measure in research?

Reduced time and resources needed for development Signup and view all the answers

What is a common disadvantage of mail surveys?

Limited control over survey administration Signup and view all the answers

Which is a crucial aspect to consider when drafting survey items for reliability?

Keeping items uniform across demographics Signup and view all the answers

What is the primary goal when piloting survey items?

To maximize reliability and validity Signup and view all the answers

When creating your own measure, which resource is useful for developing survey items?

Existing measures and theories on the research topic Signup and view all the answers

What is an advantage of conducting interviews compared to mail surveys?

Ability to capture non-verbal cues Signup and view all the answers

What should be done after drafting survey items to ensure their quality?

Pilot the items and collect feedback Signup and view all the answers

What is a potential challenge when conducting phone interviews?

Keeping sample size large and diverse Signup and view all the answers

What is one measure to ensure that survey items are valid?

Items should tap into the relevant information needed Signup and view all the answers

What should researchers consider about survey types before deployment?

Accessibility to the target population Signup and view all the answers

Study Notes

Publishing Psychology Studies

Computer scientists have been archiving their work online since the 1970s.
Physicists began archiving their work in the 1990s in a journal called “arXiv."
The American Psychological Association (APA) ran the journal Psycoloquy from 1990 to 2002.
The Association for Research in Vision and Ophthalmology (ARVO) has published the Journal of Vision since 2001.

Types of Data Collection

Online surveys are used to gather information on demographics, opinions, and beliefs, or to conduct experiments.
Examples include surveys on Qualtrics, LimeSurvey, and SurveyMonkey.
Perception/Cognition Experiments involve the presentation of stimuli to participants.
Examples of specialized programs that can run these experiments include PsychoPy and Gorilla.
Automatic test scoring can be used in both online surveys and perception/cognition experiments.

Advantages of Using the Internet for Research

Access to historical records, digital transactions, quantitative and qualitative data.
Easier recruitment and data collection.
Large and diverse samples can be accessed at low cost.
Specific groups can be targeted, including those from different countries and cultures.
Reduced experimenter bias because online experiments are standardized.
Faster data collection due to 24-hour availability.
Young people tend to be more familiar and interested in using the internet, which potentially increases motivation to participate.

Ethical Benefits

Anonymity and increased self-disclosure are benefits of using online research.
Participants may be more likely to participate due to decreased effort and cost.
Participants may feel less social pressure in online settings.

Ethical Disadvantages

Participants may suffer harm from direct participation in research.
It is crucial to consider how to protect participants in case they become distressed during online research.
Confidentiality breaches are a concern.
Facebook’s “emotional manipulation” study exemplifies a potential ethical issue with online research.

Disadvantages of Using the Internet

Sample biases and lack of generalizability can occur.
Participants may drop out of the study.
No control over the data collection setting for online research.
Online participants may invest less energy in their responses, potentially influenced by social facilitation theory.
Data may be leaked due to security issues.
Technology failures can disrupt research.
Multiple submissions from the same participants can occur resulting in inaccurate data.
Researchers need to develop more advanced technological skills to effectively conduct online research.

Threats to Data Quality

Pilot and pretest surveys to identify issues.
Collect data from a “trustworthy” source.
Manage participants effectively to avoid multiple submissions.
Establish objective exclusion criteria for participants.

Survey Methods in Research Practice

The first step is to find appropriate measures that answer your research question.
Next, determine the quality of the measure for your research purposes.
Consider whether to adapt an existing measure or create your own.
Choose the survey type and the platform to create it.
Distribute the survey and collect responses.
Determine whether the survey responses are valid.
Handle invalid responses in your dataset.

Finding Measures Using Electronic Databases

The APA PsycTEST website is a valuable resource.

Determining the Quality of a Measure

Consider its suitability for your research questions, design, target demographic, and delivery method.
Evaluate its psychometric properties, including validity and reliability.
Consider whether the questionnaire is practical based on access, cost, and ease of use.

Adapting or Creating Measures

Adapt existing measures if they are relevant to your research question and meet the quality criteria.
Ensure consistency in response scales across measures if adapting them.
Consider grouping frequency and attitude scales together.

Benefits of Using/Adapting Existing Measures

Conserves time and resources.
Easier to compare results with previous research.
Facilitates publication processes.

Creating Your Own Measure

Review existing measures.
Explore theories relevant to the topic.
Conduct focus groups to gather feedback.
Utilize resources such as the DSM or ICD.

Steps to Create a New Survey

Draft items that maximize reliability and validity.
Pilot and edit items to ensure their clarity and consistency.
Decide on the survey type and platform to create it.
Distribute the survey and collect responses.
Determine the validity of the survey responses.
Handle invalid responses in your dataset.

Survey Types

Mail Survey:
- Offers access to a wide range of participants.
- Relatively inexpensive and convenient.
- Can increase honesty in responses.
- Limited control over the administration or return of the survey.
- Requires manual data entry.
Interview:
- Suitable for all populations.
- Captures nonverbal cues.
- Keeps the interviewee focused.
- Minimizes missing data.
- High cost and time-consuming.
- Limits sample size.
- Potential to compromise anonymity.
Online Survey:
- Offers convenience, time and cost-effectiveness.
- Enables automatic collection and storage of data.
- Access to large samples globally.
- May be challenging to access populations without internet access or technical skills (e.g., the elderly, remote locations).
- Increased potential for survey fraud or incentives leading to falsification.

Creating a New Online Survey

Classify the data you intend to collect based on sensitivity level (highly sensitive, sensitive, or general).
Utilize the Data Sensitivity Calculator to determine the appropriate level.
Consider the Research Platforms Browser for suitable platforms based on data sensitivity.

Distributing and Collecting Responses

Leverage relevant organizations, such as schools, hospitals, and workplaces.
Utilize personal or professional email for dissemination.
Explore research participant pools, such as MQ SONA and Qualtrics.
Utilize social media platforms for sharing.
Employ “word of mouth” outreach.

Determining Survey Response Validity

Recognize the potential for participant inattentive or careless responding.
Identify responses from slackers, straight liners, speeders, and survey bots.
Employ methods to detect invalid responding, such as:
- Inclusion of specific items designed to detect inattention.
- Multivariate outlier analysis.
- Analyze survey response time.
- Utilize long string analyses.
- Include self-reported diligence measures.

Handling Invalid Responses

Delete all cases with inappropriate responses.
Create a cut-off score.
Conduct sensitivity analyses.

Publishing Psychology Studies

Computer scientists have been archiving their work online since the 1970s.
Physicists have been archiving their work since the 1990s in a journal called “arXiv”.
The APA published "Psycoloquy" from 1990 to 2002.
The ARVO has published the "Journal of Vision" since 2001.
Psycoloquy and Journal of Vision are examples of scholarly publications in psychology.

Types of Data Collection

Online surveys are used for collecting qualitative and quantitative data.
Common online survey software includes Qualtrics, LimeSurvey, and SurveyMonkey.
Perception and cognition experiments are conducted online.
Automatic test scoring software collects data automatically.
Software used in perception and cognition experiments includes PsychoPy and Gorilla.
Stimulus presentations and response timing data can be collected using these software programs.
Applied research, such as evaluation of therapeutic programs, is often conducted online.
An example of applied research is the MindSpot program which uses online data collection for evaluation.

Advantages of Using The Internet For Research

Historical records can be easily accessed online.
Data from digital transactions can be obtained online, such as e-commerce records.
Both quantitative and qualitative data can be collected online.
Internet research allows for easier recruitment and data collection.
Larger and more diverse samples can be recruited at lower costs.
Researchers can focus on specific groups, such as those in different countries or cultures.
There is less potential for experimenter bias in online research due to standardized procedures.
Data collection is faster due to global participation enabling 24-hour data collection.
The internet is more familiar and engaging for younger participants, which could increase motivation.

Ethical Benefits of Online Research

Anonymity and self-disclosure are often increased in online research.
Participation can be less effortful for participants, especially when completing surveys at home.
Online research reduces social pressure.

Ethical Disadvantages of Online Research

The Facebook "emotional manipulation" study raised ethical concerns. In this study, participants were exposed to either happy/positive or sad/negative posts, potentially causing harm.
Online research raises concerns about potential harm to participants, particularly when data is particularly sensitive.
Potential breaches of confidentiality are a concern in online research.

Disadvantages of Using The Internet For Research

Online samples may be biased, making it difficult to generalize findings.
Participant drop out is a concern in online research.
Researchers lack control over the data collection setting, potentially impacting results.
Participants may invest less effort in online studies, which can be explained by social facilitation theory.
Data leaks and security issues are potential risks.
Technology failures can disrupt data collection.
Multiple submissions from the same participant can negatively affect data analysis.
Researchers need to be increasingly tech-savvy to conduct online research, especially when considering software and privacy concerns.

Threats To Data Quality in Online Research

Pilot studies and pre-testing are important to improve data quality.
Data should be collected from trustworthy sources.
Effective management of participants, such as recording IP addresses, can minimize multiple submissions.
Objective exclusion criteria should be established, such as timing and attention checks, to minimize bias.

Week 10: Survey Methods in Research Practice

The first step is determining the appropriate measures to answer research questions.
The second step is evaluating the quality of a measure for a specific purpose.
The third step is deciding whether to adapt an existing measure or create a new one.
The fourth step is selecting the best type of survey and creating it.
The fifth step is distributing the survey and collecting responses.
The sixth step is evaluating the validity of survey responses.
The seventh step is addressing invalid responses in the data set.

Where to Find Measures

Electronic databases, like the APA PsycTEST website, offer a wide range of measures.

Determining The Quality of a Measure

Assess whether the measure is appropriate for the research question and design.
Consider the target demographic and the method of delivery.
Assess the psychometric properties, specifically validity and reliability.
Assess whether the measure is practical from a financial and accessibility standpoint.

Adapting Existing Measures

There are advantages to adapting an existing measure, such as saving time and money.
Adapting existing measures can make it easier to compare results with previous studies.
Reviewers may be more likely to support studies that use adapted measures.
When adapting an existing measure, consistency in response scales should be maintained.

Creating A New Measure

Create a new measure when adapting an existing measure is not possible.
When creating a new measure, existing measures, theories, focus groups, and other resources should be reviewed.

Drafting Survey Items

When drafting survey items, it is important to maximize reliability and validity.
To ensure reliability, the items should be interpreted the same way by all respondents regardless of factors like gender or time of survey completion.
Validity is ensured when the items tap into the information needed, are accurate, and are relevant to the research question.

Piloting and Editing Survey Items

Pilot and edit items to ensure reliability and validity.
Seek feedback from colleagues and supervisors.
Edit items based on feedback.
Pilot the draft survey with a small group and gather feedback.
Edit items based on pilot results.

Types of Surveys

Mail surveys are inexpensive but have low response rates and are not suitable for certain populations.
Interviews can be conducted face-to-face or over the phone and provide a more personal approach, but are more costly and time-consuming.
Online surveys are convenient, cost-effective, and offer automatic data collection but are not accessible to all populations.

Creating New Online Surveys

The first step is determining the sensitivity level of the data.
Data sensitivity can be categorized into three levels: highly sensitive, sensitive, and general.
Highly sensitive data includes information about health, criminality, genetics, race and religion, and finances.
Sensitive data includes information about personal data not related to highly sensitive categories.
General data includes anonymized data that is publicly available or aggregated.

Data Sensitivity Calculator

The data sensitivity calculator helps to determine the sensitivity level of data sets.

Research Platforms Browser

Research platforms offer tools for data storage and retrieval and are appropriate for collecting and archiving data on both general and sensitive topics.

Distributing Surveys and Collecting Responses

Distribution methods include online surveys, social media, and personal or professional email.

Detecting Invalid Responding

Measures to detect invalid responding include:
- Attention checks
- Multivariate outlier analysis
- Survey response time
- Long string analyses
- Self-reported diligence

Addressing Invalid Responses

Options for handling invalid responses include:
- Deleting all cases with inappropriate responses
- Creating a cut-off score
- Conducting sensitivity analyses

The Importance of Managing Invalid Responses

Invalid responding reduces statistical power and obscures meaningful results.

Common Invalid Responding Behaviors

There are different types of invalid responding behavior: - Slackers - Straight liners - Speeders - Survey Bots

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Week 4/5/6 + 9/10

Choose a study mode

Podcast

Questions and Answers

What is one advantage of using the internet for research?

Which of the following statements reflects an ethical disadvantage of internet research?

What issue is associated with sample biases in internet research?

What is a common method used for online data collection?

Which of the following is NOT an advantage of using the internet for research?

What was a concern regarding the Facebook emotional manipulation study?

What makes online surveys more appealing to younger participants?

What is a limitation of internet-based research settings?

What is one identified threat to data quality in research surveys?

Which step is not part of the survey research process?

What is crucial to avoid multiple submissions in survey research?

What aspect of a measure relates to its ability to produce consistent results?

When creating a survey, what is a vital factor for ensuring the measure's quality?

What might be a reason to adapt an existing measure in research?

Which method is recommended for handling invalid responses in survey data?

What is a key consideration when determining whether to create a new measure?

What is a significant drawback of online surveys?

Which type of data includes personal information related to sensitive topics?

What is one of the first steps in creating a new online survey?

What is one of the primary advantages of online surveys?

What is a potential issue that could arise from participant incentives in surveys?

Which category does data containing ecological details about endangered species fall into?

What is a recommended method for distributing a survey?

Which type of data can be shared publicly and is aggregated?

What is a key benefit of using or adapting an existing measure in research?

Which scale is associated with measuring attitudes?

What is an important focus when drafting survey items to ensure reliability?

What should be done after drafting survey items?

Which of the following is a disadvantage of using a mail survey?

What is a challenge associated with face-to-face or phone interviews?

When creating a survey, what should be considered regarding the type of survey to use?

What is a potential downside of using a mail survey?

What is the potential impact of participants engaging in highly inattentive or careless responding?

Which of the following types of invalid responses refers to selecting the same answer for each question?

What is one method to detect inattentive responding during a survey?

What is one consequence of deleting cases with inappropriate responses from a dataset?

How might one conduct a sensitivity analysis in response to suspected invalid responses?

Which statement describes 'faking good' in survey responses?

What does the term 'survey bots' refer to in the context of survey validity?

What does completing a survey in a suspiciously short amount of time indicate?

What is a characteristic of highly sensitive data?

Which type of data is NOT identifiable or re-identifiable?

What is a significant ethical benefit of conducting research via the internet?

What is a primary disadvantage of using online surveys?

Which of the following is a disadvantage unique to internet-based research?

What is a challenge posed by the lack of control over data collection settings in online research?

What step is essential before creating a new online survey?

Which of the following best describes sensitive data?

Which factor can reduce the risk of experimenter bias in online research?

What is a common concern related to participant distress in online studies?

What is a recommended approach to ensure data quality in survey research?

Which property of a measure assesses its ability to yield consistent results over time?

How can researchers prevent multiple submissions from the same participant in online surveys?

When determining the quality of a measure, which aspect evaluates its ability to measure what it intends to?

What is an important step to take after drafting survey items for a new survey?

Which factor should be considered when choosing an existing measure for adaptation?

Which survey distribution method is likely to reach a broader audience efficiently?

Which of the following is a benefit of using an existing measure in research?

What is a common disadvantage of mail surveys?

Which is a crucial aspect to consider when drafting survey items for reliability?

What is the primary goal when piloting survey items?

When creating your own measure, which resource is useful for developing survey items?

What is an advantage of conducting interviews compared to mail surveys?

What should be done after drafting survey items to ensure their quality?

What is a potential challenge when conducting phone interviews?

What is one measure to ensure that survey items are valid?

What should researchers consider about survey types before deployment?

Study Notes

Publishing Psychology Studies

Types of Data Collection

Advantages of Using the Internet for Research

Ethical Benefits

Ethical Disadvantages

Disadvantages of Using the Internet

Threats to Data Quality

Survey Methods in Research Practice