Psychology Online Research Methods

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is a potential threat to data quality that researchers should be aware of?

Technology failures (correct)

Pilot testing

Inconsistent response scales

Access to electronic databases

Which of the following is NOT a recommended solution to enhance data quality?

Conducting pilot tests

Ignoring IP addresses (correct)

Collecting data from trustworthy sources

Establishing exclusion criteria

Which factor is important when determining the quality of a survey measure?

Number of respondents needed

Cost of the survey

Validity and reliability of the measure (correct)

Duration of the survey

What should researchers do to avoid multiple submissions in online surveys?

Record IP addresses Signup and view all the answers

When is it appropriate to create a new measure instead of adapting an existing one?

When the items are confusing or poorly worded Signup and view all the answers

What is an example of a psychometric property researchers should consider for survey measures?

Internal consistency Signup and view all the answers

Which of the following describes best the handling of invalid responses in survey data?

Exclude them from all analyses Signup and view all the answers

What is a recommended practice when comparing different measures?

Keep response scales consistent Signup and view all the answers

What is a major advantage of using the internet for research?

Access to more historical records Signup and view all the answers

What ethical advantage does online research provide to participants?

Anonymity and increased self-disclosure Signup and view all the answers

What is a potential disadvantage linked to internet-based research samples?

Sample biases and lack of generalisability Signup and view all the answers

Which of the following tools is primarily used for conducting online surveys?

Qualtrics Signup and view all the answers

Which statement reflects a disadvantage of using the internet for research?

Lack of control over data collection settings Signup and view all the answers

Why might younger participants be more motivated to engage with online surveys?

The familiarity and interesting nature of the internet Signup and view all the answers

What disadvantage did the Facebook emotional manipulation study highlight?

Harm resulting from direct participation Signup and view all the answers

Which survey method is associated with automatic test scoring?

PsychoPy Signup and view all the answers

What is a primary benefit of adapting an existing measure in a study?

It takes less time and resources. Signup and view all the answers

Which of the following are scales used to measure attitudes?

Strongly disagree, Disagree, Neutral, Agree, Strongly agree Signup and view all the answers

What step involves gathering feedback on survey items to improve reliability?

Pilot and edit the items Signup and view all the answers

What is a common challenge associated with mail surveys?

They often have a poor response rate. Signup and view all the answers

Which option best describes one of the purposes of focus groups when creating a new measure?

To gather diverse perspectives and insights. Signup and view all the answers

What does validity in the context of survey development refer to?

The accuracy and relevance of the items. Signup and view all the answers

What is a disadvantage of conducting face-to-face interviews?

They limit sample size. Signup and view all the answers

Which of the following measures would most likely lead to a control over the administration of the survey?

Interviews (F2F or phone) Signup and view all the answers

What is a disadvantage of online surveys?

They increase the chance of survey fraud. Signup and view all the answers

What categorizes Highly Sensitive Data?

Data regarding race, religion, and finances. Signup and view all the answers

When considering a survey's sensitivity level, which data falls under Identifiable/re-identifiable Sensitive Data?

Cultural heritage and location information. Signup and view all the answers

What is a suitable purpose for using a data storage platform?

Archiving data safely. Signup and view all the answers

What is a consequence of respondents engaging in inattentive or careless responding?

Decreased statistical power Signup and view all the answers

What is one potential issue that can arise from offering incentives for survey participation?

They may encourage falsification of responses. Signup and view all the answers

Which of the following is NOT considered a type of invalidity in survey responses?

Self-report honesty Signup and view all the answers

Which type of data is classified as NOT Identifiable/re-identifiable?

Anonymized personal health data. Signup and view all the answers

Where should online surveys be distributed to ensure relevant responses?

Via personal professional email and relevant organizations. Signup and view all the answers

Which method can help detect inattention in survey participants?

Directly measuring response time Signup and view all the answers

What is the recommended way to deal with invalid responses in your dataset?

Create a cut-off score for attention items Signup and view all the answers

What is the first step when creating a new online survey?

Determining the sensitivity level of the data to be collected. Signup and view all the answers

What does long string analysis measure in survey responses?

Frequency of repeated responses within a block Signup and view all the answers

What is a characteristic of 'speeders' in survey responding?

They answer questions extremely quickly. Signup and view all the answers

Which factor can obscure meaningful survey results?

Careless responding Signup and view all the answers

What effect does deleting cases with inappropriate responses have on your dataset?

It reduces the power of the analysis. Signup and view all the answers

Study Notes

Publishing Psychology Studies

Computer scientists began archiving their work online in the 1970s.
Physicists followed suit in the 1990s, using a journal called "arXiv" to host their research.
Psychology publications had early online journals such as "Psycoloquy" (APA, 1990-2002) and "Journal of Vision" (ARVO, 2001 to date).

Types of Data Collection

Online surveys are used to collect data on various topics.
Examples of online survey platforms: Qualtrics, LimeSurvey, SurveyMonkey
Perception and cognition experiments can benefit from online data collection with stimulus presentation and response timing.
Examples of platforms used for this purpose: PsychoPy, Gorilla.

Advantages of Using the Internet for Research

Researchers can access historical records, digital transactions, and various quantitative and qualitative data.
This makes recruitment and data collection much easier.
It allows for accessing large, diverse samples at a lower cost.
Researchers can focus on specific groups, including those from different countries and cultures.
Online research potentially reduces experimenter bias as the standardisation removes the influence of the experimenter.
Data collection becomes faster with global participation enabling 24-hour data collection.
The internet's familiarity and engagement for younger participants increases motivation.

Ethical Benefits

Anonymity provided by online research encourages increased self-disclosure.
Online participation can be less effortful and costly for participants.
Decreased social pressure is a positive aspect of online research.

Ethical Disadvantages

The Facebook "emotional manipulation" study highlights potential harm resulting from direct participation.
Online participants may become distressed.
Breaches of confidentiality are a concern for online research.

Disadvantages of Using the Internet

Sample biases and lack of generalizability present challenges.
Drop-out rates are a concern.
Researchers have no control over the environment in which data is collected.
Participants might invest less energy in online research compared to in-person interaction.
Data security risks can lead to leaks.
Technology failures are potential obstacles.
Multiple submissions by the same participant can skew data.
Researchers need to enhance their technological expertise.

Threats to Data Quality and Solutions

Pilot testing and pre-testing are crucial to improve data quality.
Data should be collected from a trustworthy source.
Good participant management, such as recording IP addresses, helps avoid multiple submission.
Establish objective exclusion criteria (timing, attention checks) to ensure data quality.

Survey Methods in Research Practice

The first step in using survey methods is to find measures that answer your research question.
It's important to assess the quality of the measure for your purpose.
Researchers may need to adapt an existing measure or create a new one.
Choosing the appropriate survey type and platform for creation is crucial.
Distributing the survey and collecting responses is essential.
Determining the validity of survey responses is vital.
Handling invalid responses within the dataset is crucial.

Where to Find Measures

Electronic databases such as the APA PsycTEST website are excellent resources for finding measures.

Determining the Quality of the Measure

Consider whether the measure is appropriate for your research question, target demographic, and method of delivery.
Assess the measure's psychometric properties (validity and reliability).
Consider practical aspects like accessibility and cost of the measure.

Adapting or Creating Measures

When adapting an existing measure, ensure the items are clear and understandable.
Maintain consistency in response scales across measures.
Group frequency and attitude scales together to minimize confusion.

Benefits of Using or Adapting Existing Measures

Using an existing measure saves time and resources.
It allows for easier comparison with previous studies.
Reviewers are more likely to support research using existing measures.

Creating a New Measure

Conduct thorough research on existing measures, theories, and resources.
Incorporate focus group feedback.

Steps in Creating a New Measure

Drafting Survey Items:* Ensure the items are reliable and valid.
Reliability: Consider whether items are interpreted consistently across all respondents, on different days, and as intended.
Validity: Determine whether items capture the necessary information accurately and relevantly.
Piloting and Editing Items:* Collect feedback and edit items to improve quality.
Obtain feedback from colleagues or supervisors.
Pilot the draft survey with a small group.
Edit items based on feedback.

Deciding on the Survey Type and Platform

Mail Surveys: Access to target population, inexpensive, convenient.
Disadvantages: Poor response rates, not suitable for all populations, no control over administration, manual data entry.
Interviews (F2F or Phone): Suitable for all populations, captures non-verbal cues, keeps the interviewee focused, minimizes missing data.
Disadvantages: High cost, time-consuming, can limit sample size, anonymity compromised.
Online Surveys: Convenient, time and cost-effective, automatic data collection, access to large samples, global reach.
Disadvantages: May not be accessible to all populations, increased chance of survey fraud. Incentives can encourage falsification.

Creating a New Online Survey

The first step is determining the sensitivity level of the data to be collected.
Data sensitivity can be classified into three categories:
- Highly Sensitive: Data containing information about health, criminality, genetics, race and religion, and finances.
- Sensitive: Data containing information about personal information not related to factors above, cultural heritage, location, and ecological data about endangered species.
- General: Anonymized or publicly available data in aggregated form.
Use a Data Sensitivity Calculator to determine the sensitivity level of the dataset.
Choose the appropriate research platform based on the data sensitivity level.

Suitable Platforms

Highly Sensitive Data: Suitable for: collecting and safely storing survey data involving sensitive topics.
Sensitive Data: Suitable for: collecting survey data, storing data, and archiving datasets.
General Data: Suitable for: data storage, archiving data, and accessing datasets.

Distributing Surveys and Collecting Responses

Use various methods to distribute surveys:
- Collaboration with organizations (schools, hospitals, workplaces)
- Personal or professional emails
- Research participant pools (e.g., MQ SONA, Qualtrics)
- Social media
- "Word of mouth"

Determining Survey Response Validity

Inattentive or careless respondents can negatively impact data quality.
This decreases statistical power, obscures meaningful results, and can lead to spurious results.
Other forms of invalidity include:
- Faking good
- Faking bad
- Social desirability.

Detecting Invalid Responding

Include items designed to detect inattentive responding.
Conduct multivariate outlier analysis to assess statistically unlikely response patterns.
Analyze survey response time.
Utilize long string analyses to measure repeated answer selection.
Include self-reported diligence measures.

Handling Invalid Responses

Delete cases with inappropriate responses.
Create a cut-off score for attention items.
Conduct sensitivity analyses.
Replace invalid responses with plausible values.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

Explore the various ways psychology studies are published and the online tools available for data collection in research. This quiz highlights the evolution of online data collection platforms and the advantages of using the internet to access diverse samples for psychological research.