Research Methods Lecture

RESEARCH METHODS Research Fundamentals and Research Design 1  Research “an undertaking intended to extend knowledge through a disciplined inquiry and/or systematic investigation. The term “disciplined inquiry” refers to an inqu...

RESEARCH METHODS Research Fundamentals and Research Design 1  Research “an undertaking intended to extend knowledge through a disciplined inquiry and/or systematic investigation. The term “disciplined inquiry” refers to an inquiry that is conducted with the expectation that the method, results, and conclusions will be able to withstand the scrutiny of the relevant research community” (Tri-Council Policy Statement, 2022) 3 characteristics Challenges the Creative Systematic status quo 2 Characteristics of (Experimental) Science ◦ Systematic ◦ Public ◦ Peer review ◦ Empirical Testing ◦ Experimental Control (to different degrees) ◦ Probabilistic Knowledge ◦ Can never prove anything completely ◦ Replication ◦ No one study answers every question ◦ Objective ◦ Neutral 3 Model of Scientific Thought (Portney & Watkins, 2009) 4 Good theories are ◦ Account for existing data ◦ Explanatory value ◦ Predictive value ◦ Testable ◦ Parsimonious ◦ Efficient explanation ◦ Only as complicated as necessary ◦ Tentative: modified as new evidence becomes available - confirmed or not confirmed 5  Research Question/Hypotheses Makes prediction Hypothesis Implicitly assumes alternative relationships are possible Research Question No prediction Specific, focused Both Testable 6  Research Question Development Topic Problem Theory Found at the end of the Question Introduction usually 7 ETHICS IN SCIENCE 8  Intent New knowledge (unknown benefit if any; have a hypothesis) vs treatment (assumed Research benefit) vs Innovation Practice Novel practice vs customary practice Plan Uniformity vs individualized 9 Researcher Must Act with Integrity ◦ Pursue questions that are relevant and meaningful and address important issues ◦ Design research well using valid and reliable practices ◦ Carry out research completely ◦ Report results honestly and accurately ◦ Report authorship accurately ◦ On reports ◦ No plagiarism ◦ Academic Integrity - Writing Centre Resource Guide - LibGuides at Dalhousie University link to avoiding plagiarism and how to cite ◦ Manage conflicts of interest ◦ Manage resources honestly ◦ Consider consequences to society 10 Research Ethics Boards (REB) ◦ Definition ◦ Groups in an institution responsible for reviewing research proposals that will involve human subjects to determine adherence to ethical principles ◦ Roles ◦ REBs approve, reject, propose modifications to, audit, terminate research involving human subjects within an institution 11 Reviews ◦ Research involving human subjects includes: ◦ human participants, ◦ human biological material or ◦ human data ◦ Dalhousie REBs ◦ Health Sciences Research Ethics Board ◦ Social Sciences & Humanities Research Ethics Board ◦ Separate board for animal research 12 What doesn’t need an REB Review? ◦ Research that does not involve human subjects ◦ Research on living individuals based on publicly available information (e.g., newspaper articles, biographical accounts) ◦ Naturalistic observations (special case) ◦ No personal identifying information ◦ Behaviour naturally occurring ◦ Environment not staged ◦ No reasonable expectation of privacy ◦ 2ndary use of data if anonymous ◦ **Quality assurance studies**, performance reviews or testing within normal clinical or education requirements (i.e., use internal to organization) 13 REB Reviews ◦ Two rights need to be balanced: ◦ Rights and welfare of participant ◦ Right of an experimenter to seek knowledge ◦ Reviews focus upon ◦ Attainment of ethical principles ◦ Scientific merit ◦ Risk – Minimizing it & Risk-benefit ratio analysis ◦ Recruitment of participants ◦ Informed consent processes and documents ◦ Disclosure, comprehension, voluntariness, competence ◦ If deception, well justified & plans for debriefing ◦ Data storage & management - Confidentiality 14 Important Concepts ◦ Respect human rights and dignity ◦ Morally acceptable ends ◦ Morally acceptable means (methods) to those ends. ◦ Respect for person / community ◦ Free and informed consent (autonomy) ◦ Respect and protect the vulnerable ◦ Recognition of traditionally exploited groups ◦ Respect for privacy and confidentiality and anonymity where possible ◦ Risk-Benefit Analysis ◦ Benevolence ◦ Non-maleficence ◦ Just ◦ Fair ◦ Inclusive 15 Tri-Council ◦ https://tcps2core.ca/welcome ◦ Online course on research ethics recommended for all researchers involving human participants and REB members 16  RESEARCH PARADIGMS 17 Research Paradigms ◦ Quantitative ◦ Qualitative ◦ Single-subject 18 Assumptions Assumption Quantitative Qualitative Single subject Reality Single, Multiple, Single, objective constructed objective Rel’n bet Independent; Dependent; Independent researcher & researcher researcher participants does not influences influence Generalizabilit Desirable & Situation System y possible; can specific; specific; generalize to cannot cannot population generalize to generalize, too population specific Cause-effect Causal Non-causal Causal rel’n Values Value free Value bound Value free 19 Methods Method Quantitative Qualitative Single- Subject Theory A priori Grounded; A priori theory evolves with research Number & Groups, Small number, One, purposive selection of random purposive (selection of participants participant) Measurement Instruments Human Instruments tools Type of data Numerical Language Numerical Manipulation Present Absent; not Present looking to change context but understand it Control Maximized Minimized Flexible 20 Research Types ◦ Experimental vs Observational/Descriptive/Non-experimental ◦ Experimental (something has been manipulated) ◦ Experimental Group ◦ Quasi-experimental group ◦ Groups are preexisting ◦ E.g. Head injuries ◦ Single subject designs ◦ Descriptive Research (No manipulation) ◦ Developmental ◦ Normative ◦ Correlational & Predictive ◦ Qualitative ◦ Case study ◦ Describing the nature of a unique case without manipulating anything 21 Research Types ◦ Basic versus Clinical/Applied ◦ Basic research ◦ Used to develop, define, test theory ◦ Not motivated by practical application ◦ Knowledge for knowledge’s sake ◦ May have clinical/applied implications; not directly tested ◦ Clinical or applied research ◦ Directed toward an immediate practical problem 22 VARIABLES 23 Variables Experimenter manipulates Independent Active vs assigned Influenced by or co-varies with independent Dependent variable Things that vary but not the focus of the study Can affect results so try to control or Confounding/Extraneous minimize May be beyond the control of experimenter, try to account for Other testing seen in studies Status measures (neither indep or dep variable; not Inclusionary criteria Exclusionary criteria part of analyses) 24  ◦ Operationally Defined ◦ How it’s measured ◦ E.g. how are you defining reading level ◦ Reliable ◦ i.e. stable; same results today Variables tomorrow, etc. ◦ Valid ◦ Accurately represented ◦ E.g. cannot just test one aspect of language & then overgeneralise these results to full language capacity 25 EXPERIMENTAL VALIDITY 26 Experimental Validity Types ◦ Internal Validity; confidence in independent variable being the sole reason for change due to strong experimental design which maximises relationship between independent & dependent variable ◦ Construct Validity ◦ Statistical Conclusion Validity ◦ External Validity Internal validity (experiment) ◦ Degree to which the causal relationship is properly demonstrated ◦ Higher internal validity has more control over extraneous variables to eliminate or lessen their effect on DV so can state more confidently that the changes in the DV are the results of manipulation of the IV Internal Validity - Threats ◦ History ◦ External influences over time ◦ Interaction between Different Treatments (in text under construct validity) ◦ Maturation ◦ Internal changes over time ◦ Between sessions ◦ Within session Internal Validity - Threats ◦ Testing Effects ◦ Interaction between Testing and Treatment (in text under construct validity) ◦ People get better at tests as they take them; might have anxiety resulting in poorer results, etc. ◦ Instrumentation ◦ Are changes in the measuring tools influencing findings? ◦ Appropriate calibration of instruments ◦ Humans as instruments - Who collects the data ◦ Training, evidence of reliability between testers ◦ Biases – e.g., experimenter/interventionist collecting or scoring data, testers blinded? ◦ Statistical Regression to the Mean Internal Validity - Threats ◦ Participant Selection & Assignment ◦ Inclusion criteria ◦ Exclusion criteria ◦ Assignment ◦ Random assignment; works well with LARGE groups ◦ Matching; allows control for things such as age, etc. ◦ Match then random ◦ Subject Attrition(Mortality)  Internal Validity ◦ Diffusion or Imitation of Treatment - Threats ◦ Compensatory Equalization of Treatment ◦ Compensatory Rivalry or Resentful Demoralization Internal Validity (these items in text under construct validity) ◦ Measurements/instruments used ◦ Reliability and validity of measures ◦ Meaning of the variables within the study ◦ Does label match how operationalized? ◦ Construct Underrepresentation ◦ Experimenter Expectancies ◦ Participants/observers/testers ‘guess’ what experimenter wants & act accordingly ◦ Participants knowing purpose of study, group they are in changes behaviour ◦ Tester knowing group assignment ◦ Rosenthal effect – exp induces expectation in observers ◦ Placebo effect ◦ Hawthorne effect External Validity ◦ How well you can extend findings from your study to population of interest in the real world ◦ How generalizable External Validity Threats ◦ Selection of Participants ◦ Setting (Experimental Arrangements) ◦ Time ◦ Reactive Testing ◦ Multiple Treatment Interactions