Quasi Experiments PDF

Chapter 13: Quasi Experiments - Quasi Experiment= A study similar to an experiment except that the researchers do not have full experimental control (e.g., they may not be able to randomly assign participants to the independent variable conditions) - Start by selecting IV and...

Chapter 13: Quasi Experiments - Quasi Experiment= A study similar to an experiment except that the researchers do not have full experimental control (e.g., they may not be able to randomly assign participants to the independent variable conditions) - Start by selecting IV and DV - researchers might not be able to randomly assign participants to one level or the other; they are assigned by teachers, political regulations, acts of nature—or even by their own choice - Examples of Quasi-Experiments - Ex 1: Organ donation - Opt in method (people have to choose yes) vs opt out method (people must go out of way to say no) - These are called default options - People accept default because its effortless, and seems socially ok - 100% in consumed consent countries, much lower everywhere else - nonequivalent control group interrupted time-series design= A quasi-experiment with two or more groups in which participants have not been randomly assigned to groups; participants are measured repeatedly on a dependent variable before, during, and after the “interruption” caused by some event, and the presence or timing of the interrupting event differs among the groups - Ex 2: cosmetic surgery - Does cosmetic surgery really improve self esteem - Researchers cant ethically do this experimentally - Used 600 people who already had chosen to get surgery, and 250 who had originally wanted to but didn't - Looks experimental, but because of no random assignment its not - Asked people questions before and after surgery about happiness - nonequivalent control group pretest/posttest design - Ex 3: shows and suicide - Wanted to see effect of 13 reasons why - Looked at shows most popular time and examined suicide rates then - interrupted time-series design= participants are measured repeatedly on a dependent variable before, during, and after the “interruption” caused by some event, and the presence or timing of the interrupting event differs among the group - main concern is internal validity - Internal validity= the ability to draw causal conclusions from the results - Researchers do not have full control of the independent variable in a quasi-experiment - Threats to validity - Selection effects - relevant only for independent-groups designs - selection threat to internal validity applies when the kinds of participants at one level of the independent variable are systematically different from those at the other level - Design confounds - ,some outside variable accidentally and systematically varies with the levels of the targeted independent variable - Maturation threat - occurs when, in an experimental or quasi-experimental design with a pretest and posttest, an observed change could have emerged more or less spontaneously over time - History threat - occurs when an external, historical event happens for everyone in a study at the same time as the treatment - With a history threat, it is unclear whether the outcome is caused by the treatment or by the external event or factor. - Regression to the mean - occurs when an extreme outcome is caused by a combination of random factors that are unlikely to happen in the same combination again, so the extreme outcome gets less extreme over time - regression to the mean can threaten internal validity only for pretest/posttest designs - Attrition threat - attrition occurs when people drop out of a study over time - an internal validity threat when systematic kinds of people drop out of a study - Testing and instrument threats - testing threat is a kind of order effect in which participants tend to change as a result of having been tested before - A measuring instrument could change over repeated uses, and this change would threaten internal validity - Observer bias - you simply ask who measured the behaviors. Was the design blind (masked) or double-blind? For experimental demand, you can think about whether the participants were able to detect the study’s goals and respond accordingly - Demand characteristics - when participants guess what the study is about and change their behavior in the expected direction  - Placebo effect - when participants improve, but only because they believe they are receiving an effective treatment - Balancing priorities - Real world opportunities - they present real-world opportunities for studying interesting phenomena and important events - External validity - can enhance external validity because of the likelihood that the patterns observed will generalize to other circumstances and other individuals - Ethics - Many questions of interest to researchers would be unethical to study in a true experiment - experiments can be an ethical option for studying these interesting question - Construct validity - To test validity, , you would interrogate how successfully the study manipulated or measured its variable - quasi-experiments show excellent construct validity for the quasi-independent variable - STATISTICAL VALIDITY - ask how large the group differences were estimated to be (the effect size). - Quasi-Experiments and Correlational Studies - quasi-experiments seem similar in design to correlational studies - in quasi-experiments the researchers tend to select their samples more intentionally than they do in most correlational designs - In correlational studies, researchers select a sample (such as a large survey sample, as in the Cacioppo study), measure two variables, and test the relationship between them - Quasi-Independent Variables Compared with Participant Variables - participant variable= categorical variable, such as age, gender, or ethnicity, whose levels are measured rather than manipulated - Studies with participant variables are intended to document similarities and differences due to social identities - , quasi-independent variables focus less on individual differences and more on potential interventions such as laws, media exposure, or education Chapter 13 pt 2: SMALL-N DESIGNS: STUDYING ONLY A FEW INDIVIDUALS - Small-N design= a study where researchers only gather data from 1 person or animal - Each participant is treated separately  - almost always repeated-measures designs, in which researchers observe how the person or animal responds to several systematically designed conditions - Data for each individual are presented - enable us to compare each individual during treatment periods and control periods - designs are often used in therapeutic settings, to confirm that a treatment works for an individual person - Large-N design - Participants are grouped - data from an individual participant are not of interest in themselves; data from all participants in each group are combined and studied together - Data are represented as group averages - Large samples enable group averages to be estimated more precisely. - These studies are used for both basic and applied research. - Balancing priorities - Experimental Control - Case studies can effectively advance our knowledge when researchers use careful research designs - Studying Special Cases - take advantage of special medical cases - Ex: testing the man with part of his brain removed (this is very rare) - Disadvantages - Internal validity - In the case of the specific brain studies, there may be some other factors contributing to the results (ie other parts of his brain were disturbed) - external validity - may not represent the general population very well - any patient who undergoes such surgery usually has health problems not found in the general population - , we cannot be sure whether results from studies on surgery patients would apply to people with no history of epilepsy or schizophrenia - How to fix= triangulate, meaning to compare a case study’s results to research using other methods - Power of the small-N design - In educational, clinical, and work settings, practitioners can use small-N designs to learn whether their interventions work - frequently used in behavior analysis, a technique in which practitioners use reinforcement principles to improve a client’s behavior - Carefully designed small-N or single-N studies can help practitioners decide whether changes are caused by their interventions or by some other influence  - Stable baseline design= a study in which a practitioner or researcher observes behavior for an extended baseline period before beginning a treatment or other intervention - multiple-baseline design= researchers stagger their introduction of an intervention across a variety of individuals, times, or situations to rule out alternative explanations - reversal design= the other two small-N designs (stable baseline and multiple baseline), researchers observe a problem behavior both with and without treatment, but take the treatment away for a while (the reversal period) to see whether the problem behavior returns (reverses - By observing how the behavior changes as the treatment is removed and reintroduced, the researchers can test for internal validity and make a causal statement - If the treatment is really working, behavior should improve only when the treatment is applied - Reversal designs are appropriate mainly for situations in which a treatment may not cause lasting change - it may be considered harmful and unethical to withdraw an effective treatment from a patient or client. - Evaluating the Four Validities in Small-N Designs - External validity - steps to maximize the external validity of their findings - 1. triangulate by combining the results of single-N studies with other studies on animals or other groups - 2. researchers can specify the population to which they want to generalize, and they rarely intend to generalize to everyone - 3. sometimes researchers are not concerned about generalizing at all (In such cases, even if the causal statement applies only to one person, it is still useful) - construct validity - they should use multiple observers and check for interrater reliability, in case one observer is biased or the behavior is difficult to identify - statistical validity - provide enough quantitative evidence Chapter 14: Replication - Types of Replication - Direct replication=researchers repeat an original study as closely as they can to see whether the effect is the same in the newly collected data - a direct replication cannot replicate the initial study in every detail - researchers try to reproduce the original experiment as closely as possible - Conceptual replication  - researchers explore the same research question but use different procedures - The conceptual variables in the study are the same, but the procedures for operationalizing the variables are different. - replication=plus-extension=researchers replicate their original experiment and add variables to test additional questions - Why might a study not be replicable -. When a study fails to replicate, it could be an issue with the replication study itself - Even in direct replications, there are differences in sample, materials, or geography - Meta-Analysis: What Does the Literature Say? - scientific literature (or simply literature) =consists of a series of related studies, conducted by various researchers, that have tested similar variables. - literature review=researchers collect all the studies on a topic and consider them together - meta-analysis= a way of mathematically averaging the results of all the studies (both published and unpublished) that have tested the same variables to see what conclusion that whole body of evidence supports - Strengths: - Because meta-analyses usually contain data that have been published in empirical journals, you can be more certain that the data have been peerreviewed - assess the weight of the evidence in a scientific literature - Weaknesses - file drawer problem= the idea that a meta-analysis might be overestimating the true size of an effect because negligible effects or even opposite effects, have not been included in the collection process. - meta-analysis is only as powerful as the data that go into it Chapter 14 part 2: Transparency + credibility - underreporting of null findings - Researchers normally include multiple dependent variables in an experiment, especially when they are conducting exploratory research. - Sometimes only one out of a dozen variables will show a strong effect. - this practice becomes a problem if, in writing about the research, the researcher reports only the strong effects, not the weak ones - Underreporting misleads people to think the evidence for a theory is stronger than it really is. - Harking - HARKing= Hypothesizing after the results are known - Predictions that happen before data are collected are more convincing than those made after the fact, so HARKing misleads readers about the strength of the evidence - p-HACKING  - P-hacking= researchers might remove different outliers from the data, compute scores several different ways, or run a few different types of statistics - Researchers do not intentionally p-hack, but biases can creep in - practice of p-hacking is misleading when others are not told about all the different ways the data were analyzed and only the strongest version is reported - Transparent research practices - helps counter unintentional biases - Transparency helps scientists be more accountable to both themselves and the scientific community - Open data and open materials - Open science= the practice of sharing one’s data and materials freely so others can collaborate, use, and verify the results - Open data= psychologists provide their full data set, so other researchers can reproduce the statistical results or even conduct new analyses on it (increasing its usefulness) - Open materials= psychologists provide their study’s full set of measures and manipulations so others can conduct replication studies more easily - Preregistration - Preregistration= when scientists publish their study’s method, hypotheses, or statistical analyses in advance of data collection - time-stamped to help verify that they happened before data were collected - gives researchers credit for the importance of the research question and the quality of the study design—not just for the results Chapter 14 part 3: Must a study have external validity - Generalizing to Other Participants - If a study is intended to generalize to some population, the researchers must draw a probability sample from that population - population - that the population to which researchers want to generalize usually is not the population of every living person - they will specify what the population of interest is - “how” matters more than “how many.” - Sample from population doesnt mean it generalizies that population - In order to generalize to any population, you would need a probability sample of various groups - Generalizing to Other Settings - Sometimes you want to know whether a lab situation created for a study generalizes to real-world settings - ecological validity,=A study’s similarity to real-world contexts - Does a Study Have to Be Generalizable to Many People?  - 1. The best research uses random samples from the population. - 2. The best research uses people of all genders, ages, and ethnicities, and from all socioeconomic classes, regions, countries, and so on - Theory testing mode - Theory testing mode= A researcher’s intent for a study, testing association claims or causal claims to investigate support for a theory. See also generalization mode - In theory-testing mode, external validity often matters less than internal validity - generalization mode - generalization mode= when researchers want to generalize the findings from the sample in a previous study to a larger population - Cultural psychology =subdiscipline of psychology focusing on how cultural contexts shape the way a person thinks, feels, and behaves - Does a Study Have to Take Place in a RealWorld Setting? - When interrogating a study’s external validity, you ask whether the results can generalize not only to other populations but also to other settings - Field setting=When research takes place in the real world - The situation a researcher creates in a lab can be just as real as one that occurs in a restaurant or workplace. - experimental realism= They create situations in which people experience authentic emotions, motivations, and behaviors Cumulative Section: - Distinction between “basic” and “applied” research - basic= Focuses on expanding fundamental knowledge and understanding of natural phenomena or principles - applied= Aims to solve practical problems or answer specific questions with direct real-world applications - Sources of info - Primary= ontain complete descriptions of the collected data and data analyses - secondary= Secondary sources contain only summaries or interpretations - Validities - Construct validity= how well variables are measured and manipulated - Statistical validity= how well the numbers support the claim - Internal validity (12 covered in class)= in a relationship between one variable (A) and another (B), the extent to which A rather than some random other variable is responsible for B - External validity= how well does the study apply to a general population - Reliability vs. Validity in measurement - Research methods - Descriptive= designed to answer questions about the current state of affairs - Correlational= Involves the measurement of two or more relevant variables and an - assessment of the relationship between or among those variables - Pearson Product-Moment Correlation Coefficient= The most common measure of relationships among variable - Experimental=involves the active creation or manipulation of a given situation or - experience for two or more groups of individual - Hypothesis testing - Null hypothesis= Represents a default position or assumption that there is no effect, no relationship, or no difference between groups - Alpha=significance level - p-value=The calculated probability that the observed results occurred by chance, assuming the null hypothesis is true - Type I error = Rejecting the null hypothesis when it is actually true so you ou claim an effect exists when it does not - Type II errors= Failing to reject the null hypothesis when it is actually false so you miss detecting a real effect - Between design= Participants are split into separate groups, where each group experiences only one level of the independent variable - within design= he same participants are exposed to all levels of the independent variable - mixed designs= Combines elements of both between-subjects and within-subjects designs. -