week 9 & 10 notes

Publishing Psychology Studies

• Computer scientists have been archiving their work online since the 1970s.
• Physicists have been archiving their work since the 1990s in a journal called “arXiv”.
• The APA ran Psycoloquy, a journal for the publication of psychology research, from 1990 to 2002.
• Journal of Vision, run by the Association for Research in Vision and Ophthalmology (ARVO), has been publishing since 2001.

Types of Data Collection

• Online Surveys: Qualitative and quantitative data collection via platforms like Qualtrics, LimeSurvey, and SurveyMonkey.
• Perception/Cognition Experiments: Automatic test scoring and stimulus presentation, response timing via platforms like PsychoPy and Gorilla.
• Applied Research: Evaluation of therapeutic programs, for example, MindSpot, using online surveys and experiments.

Advantages of Internet Research

• Historical Records: Online data collection via the internet provides access to historical records of digital transactions and online material.
• Recruitment and Convenience: Online research allows for more accessible and efficient recruitment of participants, leading to a large and diverse sample at a lower cost than traditional methods.
• Targeted Recruitment: The internet allows researchers to specifically target particular groups, including different countries and cultures.
• Low Experimenter Bias: Online research minimizes experimenter bias as the environment is standardized and automated.
• Faster Data Collection: Global participation allows for 24-hour data collection.
• Engagement: younger generations are more familiar and comfortable with technology, potentially leading to increased motivation and participation.

Ethical Benefits of Internet Research

• Anonymity: Online research promotes anonymity, increasing self-disclosure and reducing social pressure on participants.
• Reduced Effort and Cost: The convenience of completing surveys at home reduces effort and cost for participants, potentially increasing participation rates.

Ethical Disadvantages of Internet Research

• Facebook “Emotional Manipulation” Study: This study raised concerns about the potential for harm resulting from direct participation in online research and the difficulty of addressing participant distress.
• Confidentiality Breaches: There are concerns about breaches of confidentiality in online research.

Disadvantages of Internet Research

• Sample Biases: The limitations of online access can create sample biases, affecting the generalisability of study findings.
• Drop-out: Online research has a higher rate of participant drop-out compared to traditional methods.
• Lack of Control: Researchers have less control over the data collection setting compared to traditional methods.
• Participant Effort: Participants may invest less energy in online research compared to research conducted in a controlled environment.
• Data Leakage: Increased security concerns about data leakage.
• Technology Issues: Technical issues can impact data quality and the research process.
• Multiple Submissions: Participants may submit multiple responses, creating data inaccuracies.
• Increased Technology Expertise: Researchers require greater technological expertise to conduct online research.

Threats to Data Quality

• Pilot & Pretest: Conducting pilot studies and pretesting helps ensure accurate data collection.
• Trustworthy Sources: Collecting data from trustworthy sources can help mitigate data quality concerns.
• Good Management: Recording IP addresses and implementing objective exclusion criteria help prevent multiple submissions and enhance data quality.

Week 10 - Survey Methods in Research Practice

• Step 1: Selection of Measures: Find appropriate research measures to answer research questions.
• Step 2: Measure Quality: Determine the quality of a research measure by assessing validity (construct, content, criterion) and reliability (internal consistency, test-retest reliability).
• Step 3: Adaptation vs. Creation: Decide whether to adapt an existing measure, create a new measure, or a combination of both.
• Step 4: Survey Type and Creation: Decide on the type of survey (mail, interview, online) and the platform to use.
• Step 5: Distribution and Collection: Distribute the survey and collect responses.
• Step 6: Validity Check: Determine whether the survey responses are valid.
• Step 7: Handling Invalid Responses: Address invalid responses in the dataset.

Where to Find Measures?

• Electronic Databases: APA PsycTEST website is a resource for finding suitable research measures.

Measure Quality

• Appropriateness: Assess whether a measure is appropriate for the specific research questions, target demographic, and method of delivery.
• Psychometric Properties: Examine the psychometric properties of the measure - validity and reliability.
• Practical Considerations: Ensure the measure is practical, free to use, and accessible.

Adaptation vs. Creation

• Adaptation: When adapting an existing measure, ensure that response scales remain consistent across measures and that confusing or ambiguous items are revised.
• Creation: When creating a new measure, conduct thorough research on existing measures, relevant theories, and relevant resources such as the DSM and ICD.

Draft and Pilot Survey Items

• Reliability: Ensure that items are interpreted the same way by all respondents.
• Validity: Ensure that the survey items accurately capture the information required.
• Pilot and Edit: Conduct a pilot study with a small group to gather feedback and edit items accordingly.

Survey Types

• Mail Surveys: Accessible and convenient for large populations, but have a low response rate and do not allow for control over administration or survey return.
• Interviews: Suitable for all populations but are expensive, time-consuming, and may not provide a large sample size.
• Online Surveys: Offer convenience, cost-effectiveness, automatic data collection, and access to diverse global samples, but are not accessible to individuals without internet access or tech proficiency.

Creating a New Online Survey- Data Sensitivity

• Highly Sensitive Data: Contains information about health, criminality, genetics, finances, religion, and political opinion. requires careful data handling and storage.
• Sensitive Data: Contains personal information not related to highly sensitive categories. requires careful data handling and storage.
• General Data: Anonymized or publicly available data.

Research Platforms Browser

• Suitable for: Collecting and storing sensitive data.
• Suitable for: Collecting and storing general data.

Survey Distribution and Collection

• Organisations: Distribute surveys through relevant organisations, such as schools, hospitals, or workplaces.
• Personal Email: Distribute surveys via personal or professional email.
• Research Participant Pools: Utilize research participant pools, like MQ SONA or Qualtrics.
• Social Media: Distribute surveys through social media platforms.
• Word of Mouth: Spread the word about the survey through informal channels.

Detecting and Dealing with Invalid Responses

• Inattentive Responding: 3-9% of respondents engage in inattentive responding, which can negatively affect data quality. - Identify Inattentive Respondents: Use attention checks, measure response consistency indices, and analyze response times.
• Other Forms of Invalidity: Faking good, faking bad, and social desirability bias.
  • Addressing Invalid Responses: Delete cases with inappropriate responses, apply cut-off scores, conduct sensitivity analyses, or include self-reported diligence measures.