Transparency and Credibility in Research PDF

Chapter 14 part 2: Transparency + credibility - underreporting of null findings - Researchers normally include multiple dependent variables in an experiment, especially when they are conducting exploratory research. - Sometimes only one out of a dozen variables w...

Chapter 14 part 2: Transparency + credibility - underreporting of null findings - Researchers normally include multiple dependent variables in an experiment, especially when they are conducting exploratory research. - Sometimes only one out of a dozen variables will show a strong effect. - this practice becomes a problem if, in writing about the research, the researcher reports only the strong effects, not the weak ones - Underreporting misleads people to think the evidence for a theory is stronger than it really is. - Harking - HARKing= Hypothesizing after the results are known - Predictions that happen before data are collected are more convincing than those made after the fact, so HARKing misleads readers about the strength of the evidence - p-HACKING - P-hacking= researchers might remove different outliers from the data, compute scores several different ways, or run a few different types of statistics - Researchers do not intentionally p-hack, but biases can creep in - practice of p-hacking is misleading when others are not told about all the different ways the data were analyzed and only the strongest version is reported - Transparent research practices - helps counter unintentional biases - Transparency helps scientists be more accountable to both themselves and the scientific community - Open data and open materials - Open science= the practice of sharing one’s data and materials freely so others can collaborate, use, and verify the results - Open data= psychologists provide their full data set, so other researchers can reproduce the statistical results or even conduct new analyses on it (increasing its usefulness) - Open materials= psychologists provide their study’s full set of measures and manipulations so others can conduct replication studies more easily - Preregistration - Preregistration= when scientists publish their study’s method, hypotheses, or statistical analyses in advance of data collection - time-stamped to help verify that they happened before data were collected - gives researchers credit for the importance of the research question and the quality of the study design—not just for the results