Experimental Designs Lecture 12

True experimental designs True experimental designs  Often referred to as ‘Randomized-control trials’ (RCT’s)  Follow these principles ◦ Random selection ◦ Random assignment ◦ Use of a control group ◦ Manipulation Remember that...  Random selection ◦ Is...

True experimental designs True experimental designs  Often referred to as ‘Randomized-control trials’ (RCT’s)  Follow these principles ◦ Random selection ◦ Random assignment ◦ Use of a control group ◦ Manipulation Remember that...  Random selection ◦ Is from a population ◦ Chance of being selected?  Random assignment ◦ Is within the sample ◦ Equal opportunity to be part of any group (control, experimental)  Pre-selected groups…can we do this? Using a control group  Important to determine if an effect is present  Control group often receives ‘no treatment’ Experimental group Receives treatment Simple model: 1 Creates a 2 x 2 model control – 1 treatment More complex: 1 control – Multiple treatments Including a pre-test  Important to determine that groups are equal before any intervention  But, in RCT’s, groups are randomly selected & randomly assigned Practical view of an RCT  Cross-over designs Grou Condition 1 Outcome Condition 1 Outcome pA Grou Condition 2 Outcome Condition 2 Outcome pB  Solomon designs Random Assignment to Control Group Pretest Treatment Post-test Random Assignment to experimental Pretest No Treatment Post-test Group 1 Random Assignment to experimental No Pretest Treatment Post-test Group 2 Random Assignment to experimental No Pretest No Treatment Post-test Group 3 Note use of control group Allows for many comparisons across interesting combinations RCT’s Straight RTC’s most commonly Main difficulty is Most will use an OXOX method used implementation and lack of to supply controls with time treatment Summary of designs Having a design is good…but then what? We need to make sure we are collecting Is the data we collect data that represents any good? what we wanted to measure Validity in experimental designs  Validity is... ◦ Is the treatment/test/intervention doing what it is intended to do  Back to operationalizing terms  An experiment needs to be valid to be viewed as relevant  Different types of validity exist; important to consider them when designing a study Internal & external validity  Internal ◦ How much control we have in the experiment ◦ Extraneous variables can impact our results  External ◦ How generalizable are our results  Need to consider threats and control them as much as possible  Is this a trade-off...can’t have both? Threats to internal validity Participant selection Practice effects Regression to the mean Instrumentation – subjectivity in scoring? Mortality – what about that group? Maturation – physical/psychological changes over time?  What is the threat?. A r ad iologi s t was int e r e s t e d in fi nding out whe t he r a ne w me t h od of e d uc at ing p a t ie nt s p r ior t o having and MRI would b e e ff e c t iv e in her c li e nt s. S he divid e d c li e nt s int o two gr oup s : t hos e who r e c e iv e d her ne w me t hod and t hos e who d id n’t b as e d on t he t i me of d ay t hat t he y had r e gular ap p oint me nt s ( e.g. d ay s v s e ve nings ). C lie nt s who r e c e iv e d her p r ogr am r e p or t e d le s s anx i e t y t han t ho s e who did n’t and we r e mor e lik e ly t o c omp l e t e t he t e s t. S he c onc lud e d t hat he r new me t hod was e ff e c t iv e.. C hild r e n age 5 and 6 who we r e p hys ically und er d e velop e d f or t he ir age we r e p ut on an e x p e r i me ntal d ie t ar y r e gime n. T he ir p ar e nt s we r e give n e x p lic it ins t r uc t ions as t o t he na t ur e and Threats to external validity Demand Effect of pre- Experimenter characteristic test effects s So how do we control? Match Find Control Match groups Find similar groups Use techniques that Feasible? Minimize differences? allow for controlling extraneous variables Summary so far... We’ve... Explored research generally Built up an argument through a literature review Explored the range of designs available to use Descriptive Correlational Quasi-experimental True-experimental Remember the pros and cons to each... And how different factors can affect our results!