Quasi-Experimental Designs II Lecture PDF

Quasi- experimental designs II  Meta- analysis Other Longitudin popul al ar design Cross- s sectional Sequential  Meta-analysis Not an experiment per se Consider the following: We want to discover if stereot...

Quasi- experimental designs II  Meta- analysis Other Longitudin popul al ar design Cross- s sectional Sequential  Meta-analysis Not an experiment per se Consider the following: We want to discover if stereotype threats exist in older populations. There are numerous studies that assess the impact of stereotype type threats in adults Meta-analysis: a method used to survey literature and obtain information on a combined effect of a question of interest  Meta-analysis Assume there are 20 papers that look at the topic Use a variety of designs, measures, & samples How can we determine if there is a significant impact of stereotype threats?  Meta-analysis The Box-Score method Look at population of published studies Count ones that have ‘positive’ and ‘negative’ effects Side with most ‘wins’ Are there any problematic issues with this method?  Meta-analysis Empirical method of comparison Create a common metric for each study Calculation of effect sizes for each Gives appropriate weight to each study Considers all factors in the design Conclusions about findings more telling about the effect  Powerful method to review literature Time consuming, complex method to Conclusi apply Getting simpler with ons technology Box-score method not about useful in today’s meta- scientific community analyses Allows researcher much flexibility Ethics Data collection  Meta’s close cousin Systematic/Scoping review Similar idea – to review the lit. Does not use the metrics to ‘weigh’ studies Focuses on checklist for quality to research which is used to inform conclusions  Longitudinal design Individuals followed over time Multiple time-points taken Allows identification of trends and causation  Longitudinal designs Advantages Disadvantages - Cause and effect conclusions - Highly expensive - High comparability across - Time consuming time (removes persons effects) - High attrition rates - High detail in study design/findings  Cross-sectional designs ‘Snap-shot in time’ of a phenomena Looking at individuals of different ages at one point Does not lead to cause-effect conclusions  Cross-sectional designs Advantages Disadvantages - Inexpensive - Limits comparison between - Quick to gather data groups - Low attrition rates - No information about participant development - Same age does not equal same developmental levels - Cross-generation conclusions problematic (cohort effects)  Longitudinal & Cross- sectional Both have pros and cons Longitudinal is a stronger design as it provides opportunities for cause & effect Purpose of research? The design of your study should be influenced by the question – not the preferred design of the researcher Can we combine cross-sectional & longitudinal designs into one effective technique?  Sequential designs Combines both longitudinal and cross- sectional into one study Assess individuals 20, 25, 30, etc over time Does not eliminate inherent problems with each design Retrospective interview procedure Developed in sport contexts Looks at lifestyle/past events of people Can also focus on cross-sections of participants Does not allow for cause effect conclusions But bypasses many shortcomings of both designs  Retrospective designs A quasi design: groups are pre-selected ‘Interview’ tag suggest qualitative aspects to procedure Flexibility to assess both qualitative and quantitative questions Qualitative: Why did you participate in sport at age 10; 20? Quantitative (categorical):Highest education level? Quantitative (continuous): How many hours a week did you play a give sport at ages 10, 15, etc.?  Retrospective designs: procedure Done in a one-to-one setting Similar to an interview Involves demographics, quantitative grids, qualitative aspects Interviewer guides discussion to important topics and fills in sheets/grids while participant talks Critical that interview guide be completed/tested prior  Demographics & qualitative Any variable of interest to the researcher Interviewer completes form or get participant to complete form Qualitative can be specific follow-up or probe for topic of interest  What would this look 3.00 like? 3.00 2.50 2.50 log hours (number) 2.00 2.00 activities (number) 1.50 1.50 1.00 1.00 M.A. act. 0.50 M.A. hrs 0.50 Sed. act. Sed. hrs 0.00 0.00 10--15 16--20 21--25 26--30 31--35 36--40 41--45 46--50 51--55 Age (years)  Can we recall such things from our past? What do we recall most Is this effectively? Objective vs. data Subjective variables any How far back good? can we recall? Can we test this?  Data about recall 120.00% Activities 100.00% Hours 80.00% 60.00% 40.00% 20.00% % 0.00% -20.00% -40.00% -60.00% -80.00% 10--15 16--20 21--25 26--30 31--35 36--40 41--45 46--50 51--55 Age (year) Relative differences in recall for total number of activities and hours at time 1 and time 2.  Retrospective designs The answer to all Why? Why not? our problems?  “Give a boy The law of a hammer and the hammer suddenly he will discover (Kaplan, that everything 1964) needs pounding” Conclusi Be vigilant in selecting a design ons about Quasi Each design is powerful (some designs more than others) Each design serves a (specific) goal  Understand the subtleties in each design Remember the main Conclusi criteria: preselected ons groups about Be mindful of Control Quasi the group vs. No control limitations of group designs each Cross- sectional vs. longitudinal technique Select appropriate design after study designed, not before Conclusi QUASI DESIGNS ARE VERY OFTEN USED CAUTION NEEDED IN INTERPRETATION ons about Quasi designs BE A SCEPTIC DON’T BELIEVE EVERYTHING YOU READ AND CRITICALLY EVALUATE FINDINGS