Replication Crisis in Psychology
45 Questions
0 Views

Choose a study mode

Play Quiz
Study Flashcards
Spaced Repetition
Chat to Lesson

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is a primary goal of open science practices?

  • To enhance the transparency and reliability of research (correct)
  • To increase publication rates of positive findings only
  • To support secretive research methodologies
  • To limit access to scientific data

    • What distinguishes a registered report from pre-registration of research methods?

  • Registered reports undergo peer review prior to data collection. (correct)
  • Registered reports are published before data collection begins.
  • Registered reports can be modified at any time before publication.
  • Registered reports do not require a DOI for identification.

    • What is required when pre-registering research methods?

  • A summary of previous findings
  • Detailed information on hypotheses and methodologies (correct)
  • Approval from a government agency
  • A preliminary literature review

    • What does DOI stand for in the context of pre-registration?

    <p>Digital Object Identifier (B)</p> Signup and view all the answers

    What happens to the research methods outlined in pre-registration?

    <p>They are set in stone and unchangeable. (A)</p> Signup and view all the answers

    What percentage of findings were able to be reproduced in Artner et al.'s analysis?

    <p>70% (B)</p> Signup and view all the answers

    What was one specific reason given for outcome reproducibility failure in Artner et al.'s study?

    <p>Authors did not provide enough information about methods (C)</p> Signup and view all the answers

    What is referred to as process reproducibility failure?

    <p>When reproducing findings requires a different methodology (C)</p> Signup and view all the answers

    In the Open Science Collaboration study, what was the average number of studies that replicated major findings?

    <p>36 (A)</p> Signup and view all the answers

    Which area of psychology had the highest replication rate according to the reported data?

    <p>Cognitive psychology (D)</p> Signup and view all the answers

    What was the percentage of replication reported by scientists in Baker's survey?

    <p>70% (A)</p> Signup and view all the answers

    What is outcome reproducibility failure?

    <p>When reanalysis produces different results than originally reported (B)</p> Signup and view all the answers

    What action has been increasingly required by many scientific journals prior to publication?

    <p>Data and analysis code uploads (C)</p> Signup and view all the answers

    What is the purpose of pre-registration in research methods?

    <p>To ensure transparency and improve research quality (C)</p> Signup and view all the answers

    What is included in the 'Methods' section of a pre-registration template?

    <p>Participant recruitment, selection, and compensation (D)</p> Signup and view all the answers

    Which of the following is NOT included in a pre-registration template for quantitative research?

    <p>Results and conclusions (B)</p> Signup and view all the answers

    What is a registered report?

    <p>A peer review of research methods prior to data collection (A)</p> Signup and view all the answers

    What section of a pre-registration template would include the planned analyses?

    <p>Methods (B)</p> Signup and view all the answers

    Which element is critical for ensuring ethical standards prior to research?

    <p>Conflict of Interest Statement (B)</p> Signup and view all the answers

    How should participant dropout be addressed in a research proposal?

    <p>Outlining a clear plan for handling missing data (A)</p> Signup and view all the answers

    Which aspect does not relate to a registered report's peer review process?

    <p>Statistical results interpretation (B)</p> Signup and view all the answers

    What does the null hypothesis in Null Hypothesis Significance Testing (NHST) assume?

    <p>There is no significant difference between the groups being compared. (C)</p> Signup and view all the answers

    What is a potential risk of p-hacking in research?

    <p>It can result in false-positive findings. (B)</p> Signup and view all the answers

    Which of the following is NOT an example of p-hacking?

    <p>Collecting data from all participants and reporting all findings. (C)</p> Signup and view all the answers

    How does having a large sample size (N) affect the results of studies?

    <p>It allows for trivial differences to appear significant. (C)</p> Signup and view all the answers

    According to Tukey, what is his perspective on the differences between conditions?

    <p>Effects are always different, even if only by a small margin. (D)</p> Signup and view all the answers

    Which practice could lead to a false positive without true significant effects?

    <p>P-hacking through selective reporting. (D)</p> Signup and view all the answers

    What is meant by 'dramatically different results' in research context?

    <p>Results that lead to major conclusions and implications. (B)</p> Signup and view all the answers

    What could be a reason why researchers engage in p-hacking?

    <p>To publish results that reflect what they aim to show. (D)</p> Signup and view all the answers

    What is HARKing in the context of research?

    <p>Generating new hypotheses based on results observed after the analysis. (A)</p> Signup and view all the answers

    Which of the following best describes SHARKing?

    <p>Publicly presenting post hoc hypotheses in the introduction as if they were pre-planned. (B)</p> Signup and view all the answers

    What does THARKing involve?

    <p>Presenting hypotheses from post hoc reviews in a transparent manner in the discussion section. (B)</p> Signup and view all the answers

    Why is HARKing considered unethical in scientific research?

    <p>It undermines the validity of scientific findings. (C)</p> Signup and view all the answers

    What is the potential issue with using post hoc analyses in hypothesis formulation?

    <p>They can lead to misleading conclusions if not transparent. (C)</p> Signup and view all the answers

    What outcome is associated with the first case study discussed by Hollenbeck (2017)?

    <p>Inability to replicate findings from published research. (A)</p> Signup and view all the answers

    What is indicated by an 'r' value of 0.1 in the second case study involving epidemiologists and a new drug?

    <p>A small correlation indicating minimal impact of the drug. (D)</p> Signup and view all the answers

    What is a key difference between a priori hypotheses and post hoc hypotheses?

    <p>A priori hypotheses are developed before data collection, whereas post hoc hypotheses arise after analyzing data. (B)</p> Signup and view all the answers

    What is a key feature of the registered report method compared to traditional review methods?

    <p>It includes a Stage 1 review and In-Principle Acceptance. (C)</p> Signup and view all the answers

    What evidence is associated with registered reports compared to regular articles?

    <p>They are more likely to fail to reject the null hypothesis. (A)</p> Signup and view all the answers

    What is the main disadvantage of pre-registration according to the findings?

    <p>It improves perceptions of research quality, but not replication rates. (B)</p> Signup and view all the answers

    How many stages of review do registered reports undergo?

    <p>Two stages of review. (B)</p> Signup and view all the answers

    Which method has more examples in practice, pre-registration or registered reports?

    <p>Pre-registration is more common. (A)</p> Signup and view all the answers

    What is the purpose of In-Principle Acceptance in the registered report method?

    <p>To confirm the research design before data collection. (B)</p> Signup and view all the answers

    Which statement correctly describes the limitations of registered reports?

    <p>They reduce but do not eliminate publication bias. (C)</p> Signup and view all the answers

    Why might authors prefer registered reports over traditional review methods?

    <p>They provide acceptance prior to data collection, enhancing credibility. (A)</p> Signup and view all the answers

    Flashcards

    Reproducibility Failure

    When a study's results cannot be duplicated or validated using the original data or methods.

    Process Reproducibility Failure

    Reproducing a result but needing to change the analysis method from the original study to get that result.

    Outcome Reproducibility Failure

    A study's reported significant results do not appear when reanalyzing the same data.

    Open Science Collaboration

    An effort to reproduce the results of 100 scientific studies across different laboratories.

    Signup and view all the flashcards

    Replication Rate

    The percentage of studies that successfully replicated the original findings.

    Signup and view all the flashcards

    Cognitive Psychology Replication

    Cognitive psychology studies have a 50% replication success rate.

    Signup and view all the flashcards

    Social Psychology Replication

    Social psychology studies have a 26% replication success rate.

    Signup and view all the flashcards

    Incomplete Research Data Disclosure

    A common reason for reproducibility failure, insufficient information is given about data collection methods and the analysis performed.

    Signup and view all the flashcards

    Pre-Registration of Research Methods

    A detailed plan for research methods filed online before data collection, including hypotheses, methodologies, and analyses. This plan is set in stone and cannot be changed.

    Signup and view all the flashcards

    Registered Report

    A detailed research plan submitted for peer review before data collection. High-quality protocols are provisionally accepted for publication if the authors follow the registered methodology.

    Signup and view all the flashcards

    Why Pre-register?

    Pre-registration reduces bias by preventing researchers from changing their methods after seeing the data, thus improving the reliability of research findings.

    Signup and view all the flashcards

    DOI

    A unique and permanent string of letters and numbers assigned to articles filed online, allowing easy access and identification.

    Signup and view all the flashcards

    Pre-registration for Existing Data

    Pre-registration can be applied to studies using existing or secondary data, ensuring transparency and reproducibility of analysis methods.

    Signup and view all the flashcards

    Null Hypothesis Significance Testing (NHST)

    A statistical method used to determine if there's a significant difference between groups or conditions. It assumes there is no difference.

    Signup and view all the flashcards

    P-value

    The probability of obtaining the observed results if the null hypothesis is true. A small p-value suggests evidence against the null hypothesis.

    Signup and view all the flashcards

    P-hacking

    Unethical practice of manipulating data or analysis to achieve statistically significant results.

    Signup and view all the flashcards

    False Positive

    A result that indicates an effect when there is none. Occurs when p-hacking leads to accepting a false hypothesis.

    Signup and view all the flashcards

    Data Manipulation Example

    Stopping data collection when p < .05 to ensure significance.

    Signup and view all the flashcards

    Multiple Comparisons

    Analyzing many measures but only reporting those with p < .05.

    Signup and view all the flashcards

    Covariate Addition

    Adding variables to the analysis until p < .05.

    Signup and view all the flashcards

    Participant Exclusion

    Removing participants until p < .05.

    Signup and view all the flashcards

    Why is THARKing acceptable?

    THARKing can be justified in science because it can lead to more efficient and effective research by focusing on areas that show unexpected or interesting results.

    Signup and view all the flashcards

    What's wrong with HARKing?

    HARKing is unethical because it deceives readers into thinking the hypothesis was a priori, which can lead to misleading scientific conclusions.

    Signup and view all the flashcards

    Case Study 1: HARKing?

    A researcher created long questionnaires and found significant correlations. These findings were presented as a priori hypotheses, but the hypothesis actually came after analyzing the data.

    Signup and view all the flashcards

    Case Study 2: HARKing?

    Researchers found a small correlation between a drug and survival rate, but they noticed a difference between male and female patients. Some researchers may try to present this difference as an a priori hypothesis.

    Signup and view all the flashcards

    How can you avoid HARKing?

    Formulate hypotheses based on existing research and theory before collecting any data. This ensures that the research process proceeds in a logical, scientific manner.

    Signup and view all the flashcards

    Preregistration

    A detailed plan for a research study, outlining methods, hypotheses, and analyses, filed online before data collection.

    Signup and view all the flashcards

    PRP-QUANT

    A template for preregistering quantitative psychology research, emphasizing transparency and reproducibility.

    Signup and view all the flashcards

    Data Accessibility Statement

    A section in a preregistration plan that outlines how and where the study data will be made available to other researchers.

    Signup and view all the flashcards

    Time Point of Registration

    The date and time when the research plan was preregistered, providing a timestamp for the protocol.

    Signup and view all the flashcards

    IRB Status

    A section in a preregistration plan that indicates whether the research protocol has been approved by the Institutional Review Board (IRB).

    Signup and view all the flashcards

    Stage 1 Review

    The initial review of a Registered Report's proposed methodology and research plan.

    Signup and view all the flashcards

    In-Principle Acceptance

    The approval of a Registered Report's methodology and research plan, guaranteeing publication if the research is conducted as planned.

    Signup and view all the flashcards

    Stage 2 Review

    The final review of a Registered Report after the research is completed, verifying if the methods were followed accurately.

    Signup and view all the flashcards

    Pre-Registered Report

    A research plan that includes methods and hypotheses but doesn't undergo a formal Stage 1 review.

    Signup and view all the flashcards

    Traditional Review

    The traditional peer review system that occurs after a research study has been completed.

    Signup and view all the flashcards

    Pre-registration vs. Registered Report

    Pre-registration only plans the study beforehand. Registered Reports also have a Stage 1 review and 'In-Principle' acceptance, guaranteeing publication if the study is conducted according to the plan.

    Signup and view all the flashcards

    Publication Bias

    The tendency for studies with statistically significant results to be published more often, skewing the scientific literature.

    Signup and view all the flashcards

    Study Notes

    Replication Crisis in Psychology

    • The replication crisis highlights the challenge of consistently replicating research findings in psychology.
    • Replicability refers to the ability to repeat a study's findings in a different lab with similar methods.
    • The crisis arises when published research findings cannot be repeated.
    • Replicability is distinct from reproducibility. Reproducibility is the ability for another researcher to replicate analyses using the original data and code.

    Causes of Reproducibility Failure

    • Process reproducibility failure occurs when the original analysis cannot be replicated due to missing data, code, necessary software, or tools.

    • Outcome reproducibility failure happens when a reanalysis produces different results from the original study, potentially due to errors in either the original study or the replication attempt.

    • Factors contributing to replication failures include publication pressures, insufficient data or analyses, and insufficient research methods.

    Case Study Examples of Failed Replication

    • Studies like the Stanford Prison Experiment have been criticized for methodological and ethical concerns.
    • The "bystander effect," suggesting reduced help in emergencies when others are present, has seen mixed replication results.
    • Theories concerning stereotype threat and its effects on underperformance have experienced mixed replication.
    • Concepts in cognitive psychology and developmental psychology like spotlight attention, the dual-process model of memory, the mirror neuron system, and the Mozart Effect have faced challenges in replication.
    • Clinical psychology theories have had mixed success with replications regarding the Dodo bird verdict, the power of optimism on health outcomes, and the efficacy of memory recovery techniques.
    • Neuroscientific studies regarding the amygdala and left-brain/right-brain distinction have had variations reported.

    Statistical Assumptions and Practices

    • Misunderstanding statistical concepts like null hypotheses or the meaning of p-values can lead to erroneous conclusions.
    • Small sample sizes reduce power, meaning it becomes harder to detect true effects.
    • Publication bias favors the publication of studies with significant findings, often overlooking non-significant or null results.
    • P-hacking is the unethical practice of manipulating statistical analyses to achieve significant results, often leading to false positives.
    • Cherry-picking data involves selectively reporting only the data supporting the hypothesis, while ignoring contradictory data.

    Proposed Solutions for the Replication Crisis

    • Pre-registration: involves detailing research methods, hypotheses, and analysis plans before data collection, increasing transparency and reducing potential bias.
    • Registered reports: involve peer review of the research protocol before data collection, making the methodology more scrutinized.
    • Open science practices: like open data and open-source methodologies can increase transparency and aid replication attempts.

    Role of Social/Cultural Norms

    • Scientific policies must also be adopted to foster changes in behavior around data analysis and replication.

    Review Methods

    • Systematic reviews assess the validity of research methods to find evidence on a topic based on pre-defined criteria without combining the results statistically.
    • Meta-analyses statistically combine the results of multiple studies to examine a similar research question and potentially answer findings definitively.

    Future Directions

    • Future research should focus on open-source methods and web-based data sources for more transparent, accessible research.

    Studying That Suits You

    Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

    Quiz Team

    Related Documents

    Replication Studies in Psychology PDF

    Description

    Explore the replication crisis in psychology, focusing on the challenges of reproducibility and the implications for research findings. Understand the differences between replicability and reproducibility, as well as the causes of reproducibility failures. This quiz is essential for anyone interested in the integrity of psychological research.

    More Like This

    Use Quizgecko on...
    Browser
    Open
    Browser
    Browser