Replication Crisis in Psychology
45 Questions
0 Views

Choose a study mode

Play Quiz
Study Flashcards
Spaced Repetition
Chat to lesson

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is a primary goal of open science practices?

  • To enhance the transparency and reliability of research (correct)
  • To increase publication rates of positive findings only
  • To support secretive research methodologies
  • To limit access to scientific data

    • What distinguishes a registered report from pre-registration of research methods?

  • Registered reports undergo peer review prior to data collection. (correct)
  • Registered reports are published before data collection begins.
  • Registered reports can be modified at any time before publication.
  • Registered reports do not require a DOI for identification.

    • What is required when pre-registering research methods?

  • A summary of previous findings
  • Detailed information on hypotheses and methodologies (correct)
  • Approval from a government agency
  • A preliminary literature review

    • What does DOI stand for in the context of pre-registration?

    <p>Digital Object Identifier</p> Signup and view all the answers

    What happens to the research methods outlined in pre-registration?

    <p>They are set in stone and unchangeable.</p> Signup and view all the answers

    What percentage of findings were able to be reproduced in Artner et al.'s analysis?

    <p>70%</p> Signup and view all the answers

    What was one specific reason given for outcome reproducibility failure in Artner et al.'s study?

    <p>Authors did not provide enough information about methods</p> Signup and view all the answers

    What is referred to as process reproducibility failure?

    <p>When reproducing findings requires a different methodology</p> Signup and view all the answers

    In the Open Science Collaboration study, what was the average number of studies that replicated major findings?

    <p>36</p> Signup and view all the answers

    Which area of psychology had the highest replication rate according to the reported data?

    <p>Cognitive psychology</p> Signup and view all the answers

    What was the percentage of replication reported by scientists in Baker's survey?

    <p>70%</p> Signup and view all the answers

    What is outcome reproducibility failure?

    <p>When reanalysis produces different results than originally reported</p> Signup and view all the answers

    What action has been increasingly required by many scientific journals prior to publication?

    <p>Data and analysis code uploads</p> Signup and view all the answers

    What is the purpose of pre-registration in research methods?

    <p>To ensure transparency and improve research quality</p> Signup and view all the answers

    What is included in the 'Methods' section of a pre-registration template?

    <p>Participant recruitment, selection, and compensation</p> Signup and view all the answers

    Which of the following is NOT included in a pre-registration template for quantitative research?

    <p>Results and conclusions</p> Signup and view all the answers

    What is a registered report?

    <p>A peer review of research methods prior to data collection</p> Signup and view all the answers

    What section of a pre-registration template would include the planned analyses?

    <p>Methods</p> Signup and view all the answers

    Which element is critical for ensuring ethical standards prior to research?

    <p>Conflict of Interest Statement</p> Signup and view all the answers

    How should participant dropout be addressed in a research proposal?

    <p>Outlining a clear plan for handling missing data</p> Signup and view all the answers

    Which aspect does not relate to a registered report's peer review process?

    <p>Statistical results interpretation</p> Signup and view all the answers

    What does the null hypothesis in Null Hypothesis Significance Testing (NHST) assume?

    <p>There is no significant difference between the groups being compared.</p> Signup and view all the answers

    What is a potential risk of p-hacking in research?

    <p>It can result in false-positive findings.</p> Signup and view all the answers

    Which of the following is NOT an example of p-hacking?

    <p>Collecting data from all participants and reporting all findings.</p> Signup and view all the answers

    How does having a large sample size (N) affect the results of studies?

    <p>It allows for trivial differences to appear significant.</p> Signup and view all the answers

    According to Tukey, what is his perspective on the differences between conditions?

    <p>Effects are always different, even if only by a small margin.</p> Signup and view all the answers

    Which practice could lead to a false positive without true significant effects?

    <p>P-hacking through selective reporting.</p> Signup and view all the answers

    What is meant by 'dramatically different results' in research context?

    <p>Results that lead to major conclusions and implications.</p> Signup and view all the answers

    What could be a reason why researchers engage in p-hacking?

    <p>To publish results that reflect what they aim to show.</p> Signup and view all the answers

    What is HARKing in the context of research?

    <p>Generating new hypotheses based on results observed after the analysis.</p> Signup and view all the answers

    Which of the following best describes SHARKing?

    <p>Publicly presenting post hoc hypotheses in the introduction as if they were pre-planned.</p> Signup and view all the answers

    What does THARKing involve?

    <p>Presenting hypotheses from post hoc reviews in a transparent manner in the discussion section.</p> Signup and view all the answers

    Why is HARKing considered unethical in scientific research?

    <p>It undermines the validity of scientific findings.</p> Signup and view all the answers

    What is the potential issue with using post hoc analyses in hypothesis formulation?

    <p>They can lead to misleading conclusions if not transparent.</p> Signup and view all the answers

    What outcome is associated with the first case study discussed by Hollenbeck (2017)?

    <p>Inability to replicate findings from published research.</p> Signup and view all the answers

    What is indicated by an 'r' value of 0.1 in the second case study involving epidemiologists and a new drug?

    <p>A small correlation indicating minimal impact of the drug.</p> Signup and view all the answers

    What is a key difference between a priori hypotheses and post hoc hypotheses?

    <p>A priori hypotheses are developed before data collection, whereas post hoc hypotheses arise after analyzing data.</p> Signup and view all the answers

    What is a key feature of the registered report method compared to traditional review methods?

    <p>It includes a Stage 1 review and In-Principle Acceptance.</p> Signup and view all the answers

    What evidence is associated with registered reports compared to regular articles?

    <p>They are more likely to fail to reject the null hypothesis.</p> Signup and view all the answers

    What is the main disadvantage of pre-registration according to the findings?

    <p>It improves perceptions of research quality, but not replication rates.</p> Signup and view all the answers

    How many stages of review do registered reports undergo?

    <p>Two stages of review.</p> Signup and view all the answers

    Which method has more examples in practice, pre-registration or registered reports?

    <p>Pre-registration is more common.</p> Signup and view all the answers

    What is the purpose of In-Principle Acceptance in the registered report method?

    <p>To confirm the research design before data collection.</p> Signup and view all the answers

    Which statement correctly describes the limitations of registered reports?

    <p>They reduce but do not eliminate publication bias.</p> Signup and view all the answers

    Why might authors prefer registered reports over traditional review methods?

    <p>They provide acceptance prior to data collection, enhancing credibility.</p> Signup and view all the answers

    Study Notes

    Replication Crisis in Psychology

    • The replication crisis highlights the challenge of consistently replicating research findings in psychology.
    • Replicability refers to the ability to repeat a study's findings in a different lab with similar methods.
    • The crisis arises when published research findings cannot be repeated.
    • Replicability is distinct from reproducibility. Reproducibility is the ability for another researcher to replicate analyses using the original data and code.

    Causes of Reproducibility Failure

    • Process reproducibility failure occurs when the original analysis cannot be replicated due to missing data, code, necessary software, or tools.

    • Outcome reproducibility failure happens when a reanalysis produces different results from the original study, potentially due to errors in either the original study or the replication attempt.

    • Factors contributing to replication failures include publication pressures, insufficient data or analyses, and insufficient research methods.

    Case Study Examples of Failed Replication

    • Studies like the Stanford Prison Experiment have been criticized for methodological and ethical concerns.
    • The "bystander effect," suggesting reduced help in emergencies when others are present, has seen mixed replication results.
    • Theories concerning stereotype threat and its effects on underperformance have experienced mixed replication.
    • Concepts in cognitive psychology and developmental psychology like spotlight attention, the dual-process model of memory, the mirror neuron system, and the Mozart Effect have faced challenges in replication.
    • Clinical psychology theories have had mixed success with replications regarding the Dodo bird verdict, the power of optimism on health outcomes, and the efficacy of memory recovery techniques.
    • Neuroscientific studies regarding the amygdala and left-brain/right-brain distinction have had variations reported.

    Statistical Assumptions and Practices

    • Misunderstanding statistical concepts like null hypotheses or the meaning of p-values can lead to erroneous conclusions.
    • Small sample sizes reduce power, meaning it becomes harder to detect true effects.
    • Publication bias favors the publication of studies with significant findings, often overlooking non-significant or null results.
    • P-hacking is the unethical practice of manipulating statistical analyses to achieve significant results, often leading to false positives.
    • Cherry-picking data involves selectively reporting only the data supporting the hypothesis, while ignoring contradictory data.

    Proposed Solutions for the Replication Crisis

    • Pre-registration: involves detailing research methods, hypotheses, and analysis plans before data collection, increasing transparency and reducing potential bias.
    • Registered reports: involve peer review of the research protocol before data collection, making the methodology more scrutinized.
    • Open science practices: like open data and open-source methodologies can increase transparency and aid replication attempts.

    Role of Social/Cultural Norms

    • Scientific policies must also be adopted to foster changes in behavior around data analysis and replication.

    Review Methods

    • Systematic reviews assess the validity of research methods to find evidence on a topic based on pre-defined criteria without combining the results statistically.
    • Meta-analyses statistically combine the results of multiple studies to examine a similar research question and potentially answer findings definitively.

    Future Directions

    • Future research should focus on open-source methods and web-based data sources for more transparent, accessible research.

    Studying That Suits You

    Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

    Quiz Team

    Related Documents

    Replication Studies in Psychology PDF

    Description

    Explore the replication crisis in psychology, focusing on the challenges of reproducibility and the implications for research findings. Understand the differences between replicability and reproducibility, as well as the causes of reproducibility failures. This quiz is essential for anyone interested in the integrity of psychological research.

    More Like This

    Replication
    106 questions

    Replication

    PositiveDenouement avatar
    PositiveDenouement
    Réplication de l'ADN: Procaryotes vs Eucaryotes
    77 questions

    Réplication de l'ADN: Procaryotes vs Eucaryotes

    WarmSelenium7607 avatar
    WarmSelenium7607
    Replication Factor in Distributed Systems
    10 questions

    Replication Factor in Distributed Systems

    AppreciableEpitaph avatar
    AppreciableEpitaph
    Psychology About Questionable Research Practices
    20 questions

    Psychology About Questionable Research Practices

    QuietGrossular1828 avatar
    QuietGrossular1828
    Use Quizgecko on...
    Browser
    Open
    Browser
    Browser