Ch.2 PDF - Reading and Evaluating Scientific Research

READING AND EVALUATING SCIENTIFIC RESEARCH  Scientific Measurements: Objectivity Objective measurements: are the measure of an entity or behaviour that, within an allowed margin of error, is consistent across instruments and observers Variable: refers to the object, concept, o...

READING AND EVALUATING SCIENTIFIC RESEARCH  Scientific Measurements: Objectivity Objective measurements: are the measure of an entity or behaviour that, within an allowed margin of error, is consistent across instruments and observers Variable: refers to the object, concept, or event being measured. Behavioural measures Magnetic resonance imaging (MRI) Blood or saliva Self-reporting Operational Definitions Operational definitions: Statements that describe the procedures (or operations) and/or specific measures that are used to record observations Reliability Reliability: is when a measure provides consistent and stable answers across multiple observations and points in time Test-retest reliability Alternate-forms reliability Inter-rater reliability Same measurement Validity Validity: is the degree to which an instrument or procedure actually measures what is claims to measure Generalizability of Results Generalizability: refers to the degree to which one set of results can be applied to other situations, individuals, or events § Study large groups § Sample to Population § Critical evaluation of findings § Beware of cases of over-generalization § “Mozart effect”  Generalizability of Results Sample type § Random sample: every individual of a population has an equal chance of being included § Convenience sample: samples of individuals who are the most readily available Location of study § Laboratory vs naturalistic research § Ecological validity: the degree to which the results of a laboratory study can be applied to or repeated in the natural environment SOURCES OF BIAS Sources of Bias in Research Researcher Bias vs Subject Bias Hawthorne effect: a term used to describe situations in which behaviour changes as a result of being observed  Demand Characteristics and Participant Behaviour Demand characteristics: inadvertent cues given off by the experimenter or the experimental context that provide information about how participants are expected to behave Clever Hans effect  Clever Hans Investigation Isolated Hans and questioner from any spectators Using people other than his owner to ask the questions Tested Hans with blinders Varied whether the questioner knew the answer to the question in advance Social Desirability Responding Social desirability responding: research participants respond in ways that increase the chances they will be viewed favourably Can minimize through assurances of anonymous/confidential questioning Yuuuuge! Observer expectancy Observer Expectancy effect: Researcher’s expectations can influence subject’s behaviour Teacher ‘favouritism’ ‘Bright’ rats vs. ‘dull’ rats  Placebo effect Placebo effect: a measurable and experienced improvement in health or behaviour that cannot be attributable to a medication or treatment “All in their head” or actual physiological response? § Some evidence of physiological pain relief and changes in brain activation § Conditioning effects § Nocebo effect REDUCING BIAS Techniques That Reduce Bias 1. Anonymity: each individual’s responses are recorded without any name or other personal information that could link a particular individual to specific results 2. Confidentiality: means the results will only be seen by the researcher 3. Inform participants Reduces participant’s anxiety and social desirability bias Techniques That Reduce Bias 4. Single-blind study: the participants do not know the true purpose of the study, or else do not know which type of treatment they are receiving (e.g., placebo or treatment drug) 5. Double-blind study: a study in which neither the participant nor the experimenter knows the exact treatment for any individual Not literally blind!  Sharing the Results Academic journals § Peer review: is a process in which papers submitted for publication in scholarly journals are read and critiqued by experts in the specific field of study § Replication: the process of repeating a study and finding a similar outcome each time  The Replication Crisis Chance results can and do happen Publication bias results in studies with affirmative evidence being more likely to be accepted into academic journals Which should we believe? The original report or the failed replication § Studies can fail to be replicated for any number of reasons § We must base our beliefs on the entirety of the evidence Weak forms of evidence Anecdotal evidence: an individual’s story or testimony about an observation or event that us used to make a claim as evidence Appeal to authority: the belief in an “expert’s” claim even when no supporting data or scientific evidence is present Biased expert? Appeal to common sense Tradition or novelty Weak forms of evidence Selective use of data § Statistics are often inappropriately used to bolster weak arguments Popularized by  TYPES OF RESEARCH: DESCRIPTIVE  Descriptive Research Descriptive data § From observations § No attempt to explain the ‘why’ Generated from: 1. Case studies 2. Naturalistic observation 3. Surveys and questionnaires  Descriptive Research 1. Case studies: are in-depth reports about the details of a specific case Difficult to generalize findings E.g., Phineas Gage – Drastic personality changes  Descriptive Research 2. Naturalistic observation: when psychologists unobtrusively observe and record behaviour as it occurs in the subject’s natural environment 3. Surveys and questionnaires Participant makes the observations  TYPES OF RESEARCH: CORRELATIONAL  Correlational Research Correlational research: involves measuring the degree of association between two variables Correlations have: 1. Direction: Positive vs negative 2. Magnitude: Correlation coefficient (-1 to +1) Correlational Research ***Correlation does NOT equal causation*** Third variable problem Illusory Correlations Illusory Correlations  Illusory Correlations Illusory correlations: relationships that really exist only in the mind rather than in reality – Crime increases when the moon is full – Opposites attract – Gamblers on a “hot streak” – Stereotypes  TYPES OF RESEARCH: EXPERIMENTAL Experimental Research Experimental group: receives special treatment in regard to the IV Control group: similar subjects who do not receive the special treatment Random Assignment: a technique for dividing samples into two or more groups Experimenter Control: variables are manipulated Experimental Research Independent Variable: Presumed cause Dependent Variable: Presumed effect Confounding variables: variables outside of the researchers control that might affect the results  Experimental Research Between-subjects design: Control Experimental Participants who are in different groups are compared vs. § A large sample and random assignment makes equal groups likely, but not guaranteed Within-subjects design: All participants respond to all types of Control Experimental stimuli or experience all experimental conditions § Order effects vs. § Separating measurements in time § Counterbalancing Quasi-Experimental Method Quasi-experimental research: a research technique in which two or more groups that are compared are selected based on predetermined characteristics, rather than random assignment E.g., Comparing men and women Cannot determine cause-and-effect RESEARCH ETHICS Promoting the Welfare of Research Participants Ethical issues must be addressed before a study begins – Tuskegee Syphilis Study Research Ethics Board (REB): a committee of researchers and officials at an institution charged with the protection of human research participants  Weighing the Risks and Benefits of Research Potential stress to participants might include § Physical stress § Cognitive and emotional stress § Writing about traumatic experiences § Stress usually minor; benefits need to outweigh risks  Obtaining Informed Consent Informed consent: a potential volunteer must be informed of the purpose, tasks, and risks involved in the study, and give consent to participate based on the information provided Topic Nature of stimuli Nature of tasks Duration Risks Steps taken to minimize risks Obtaining Informed Consent Deception: misleading or only partially informing participants of the true topic or hypothesis under investigation – Given enough information to consent Full consent – Can refuse participation without fear of penalty – Given equal opportunities – The right to withdraw – Withhold responses Debriefing: means that the researchers should explain the true nature of the study, and especially the nature of and reason for the deception  The Welfare of Animals in Research Animals used when: § Treatments cannot be applied to humans § Heritability studies require species with short lifespans § Examining evolutionary origins of behaviour and cognition Committees oversee ethical treatment § Given appropriate housing, feeding, and sanitation § Studies of sick or stressed animals do not provide generalizable results § Risk and discomfort needs to be justified and managed humanely  Ethical Collection, Storage, and Reporting of Data Data kept for 3 to 5 years § Replication Honesty with data paramount § Acknowledge conflicts of interest § Scientific misconduct has lasting repercussions § The unfounded panic surrounding vaccines and autism BASIC STATISTICS Descriptive Statistics Descriptive statistics: are a set of techniques used to organize, summarize, and interpret data – Frequency – Central tendency – Variability Frequency Frequency: the number of observations that fall within a certain category or range of scores Normal Distribution Negatively skewed Positively skewed distribution distribution Central Tendency Central tendency: a measure of the central point of a distribution (mean, median, mode) Variability Variability: the degree to which scores are dispersed in a distribution Variability Standard deviation: a measure of variability around the mean 75% ± 10% This means that 68% of the class scored between 65% and 85% HYPOTHESIS TESTING Hypothesis Testing HYPOTHESIS TESTING: FOLLOW-UP Hypothesis Testing Hypothesis test: a statistical method of evaluating whether differences among groups are meaningful, or could have been arrived at by chance alone Statistical significance: implies that the mean of the groups are farther apart than you would expect them to be by random chance alone The Limits of Statistical Hypothesis Testing If we have a p-value of 0.05 (a common standard), this means that there is less than a 5% chance the difference between the two groups was due to chance § What if conducted 100 tests? Acceptable p-value is arbitrary  The Limits of Statistical Hypothesis Testing Testing more people, makes significant results more likely, even for meaningless differences § Statistically significant does not necessarily mean practical significance Effect size as a more nuanced alternative § Calculated value indicates the degree of the difference between groups rather than reducing the significance decision to a yes/no decision