Research Methods – Final Exam Notes PDF

RESEARCH METHODS – final exam Professional Development - Career options - What can you do with a psychology degree: - Jobs - Human resource, market research, immigration officer, government researcher, advertising, case wor...

RESEARCH METHODS – final exam Professional Development - Career options - What can you do with a psychology degree: - Jobs - Human resource, market research, immigration officer, government researcher, advertising, case worker, sales rep, media buyer, travel agent, youth worker, administration, counsellor, health services, cultural diversity consultant, etc. - Professional program - Skills employers want: - Communication skills - Clear writing - Persuasive speaking - Careful listening - Strong work ethic - High ethical standards - Effective time management - Sense of initiative - Persistence in the face of challenges - Can plan and carry out projects - Teamwork skills - Working productively with others - Interpersonal skills - Deals well with a wide variety of people - Good conflict management - Graduate Programs - Master’s program - Terminal master’s - Not apart of phD program - E.g. counselling, social work - Interactions with people based - phD programs - 5-7 years combined with MA - Clinical or non-clinical - everything else - Research based - Job examples with a graduate degree - Addictions counselor  - Marriage and family therapist - School councillor - University professor - I/O psychologist - Clinical psychologist - Academic success - Reading for comprehension - Distraction free setting - Take notes to summarize key points - Look up unknown terms - Studying effectively - Connect the concepts to each other and to things you care about - Organized ideas are easier to remember than lists of unrelated facts - Familiarity is key - Test yourself and others - Flash cards - Spaced practise is better than cramming - Time management - Break tasks into smaller ones - Be aware of the planning fallacy - Block more time than you think you’ll need for each deadline - Plan difficult tasks based on your most productive time of day - Caring for yourself - Each 1 hour of class = 2-3 hours of work - Set aside and protect time when your not working\ - Take a break when you are tired - Use services - Learning skills service - Financial counselling - FRESH - nutrition education program - Psychological services - Build and academic network - Office hours/ get to know professors - Voice your opinions - Class discussions - Attending departmental events - Find out about talks on websites - How to join research labs - Choose a lab based on the clusters of research - Clinical science and psychopathology (CSP) - Cognitive, developmental, and brain sciences (CDB) - Industrial/ organizational psych. (I/O) - Social, personality and developmental psych (SPDP)  - You can email professors with intriguing research and ask if they volunteers - Use proper email format and tell them your looking to join as a “research assistant” - Research assistants have 6-12 hrs/week of tasks - Independence and responsibility increases as you stay in the lab longer - Regular lab or team meetings - Tell them why that specific lab and your availability - Attach resume and grade report to email - You can apply to work in labs as a work/study student - You can do your own research (upper years) - Why to join research labs - Better insight into the research progress - Find out if research is a good career path for you - Meet like minded students - Get to know potential mentors - Grad students, post-docs, the PI - Attending conferences - Colloquium: when a professor from another university comes to present - Usually holds a reception afterwards - Job talk: when a candidate for a tenure-track job presents their research - Advantages - Opportunities - Advocacy - Mentoring and support - You can register to any conference - Attending talks that interest you - Talk to researchers at poster sessions and social events - Professional Organizations - Canadian psychological association - American psychological association - Association for psychological science - Graduate school - Low structure, self-motivated, independent work - Delaying entry to the workforce - Keep in mind for right now!! - Choose relevant courses - Range of psychology courses  - Take advanced courses in your area - Stake stats and lab courses (programming bonus) - Get some lab experience - Get to know your professors (2-3 letters needed) - Get good grades (at least 3.5) - After applying - summer/fall of year you apply - Write the Graduate Record Exam (GRE) - Standardized test for all areas of grad school - Includes - Verbal: vocabulary, comprehension - Quantitative: algebra, geometry, other math - Analytical: two essay questions - Weighted around the same as your GPA - Plan to write it in the summer - Contact potential supervisors and schools - School should be - Reputable for your field - In a location you can live - Willing to fund you - Supervisor should be - Compatible with research interests - Available to mentor your - Good at what they do - To contact supervisors - Mention your interest in the program - Talk about why you want to work with them - Ask if they are looking for potential students - Attach your CV with GPA and GRE scores - Get reference letters - Pick professors who know you well and make it easy for them - Ask them if they would be willing to write the letters - Once they agree give them a package with all relevant information - Write your professional statement - An academic, professional word document that communicates as forcefully as possible why your a good fit for the program (sell yourself) - It should explain - What you want to study - Why you want to study it - What relevant experience you have  - What you plan to do with your degree once you have it - Show your strengths by having correlating experiences and accomplishments - Tailor each statement to the specific program, follow any specific guidelines, and don't exceed word limit Survey Research - Uses self-report - People report on their own thoughts, feelings, behaviors, etc. - Attempts to obtain generalizable samples - Ideally large and random - Types - Interviews - Structured or unstructured - Costly - Interviewer bias - Social desirability concerns - Phone surveys - Structured or unstructured - It used to be easy to get random samples yet now with telemarketing and cell phones it doesn't anymore - Cheaper - Fewer social desirability concerns - Questionnaires - Paper or electronic - Cheapest - Fewest social desirability concerns - Survey advantages - Can assess non-observable variables as well as variables that you cannot (ethically) manipulate - Demographic information (sex, age, ethnicity) - Attitudes and beliefs - Past behavior - Current behavior that cannot be observed - Motivation and emotion - Personality traits - Easy to administer  - Quick to administer and score - Can gather a lot of information - Requires few resources - Survey Disadvantages - Accuracy may be low - Participants may lack insight about certain variables - May forget previous behavior - May respond in a socially desirable manner (e.g. lying) - Not manipulating IV, thus cannot demonstrate causation - True of all correlational/ non-experimental research - Questionnaire design - Must develop a valid survey question - Each item should be BRUSO: - Brief - Avoid: - Long or run-on sentences - Unnecessary words - Technical terms, acronyms, jargon - Relevant - Avoid the temptation to include lots of extra items - Especially personal or nosy questions - Unambiguous - Avoid - Vague or imprecise terms - (e.g. “health problems”, “psychological issues”) - Negative wording - (“do you disagree that x?”) - Specific - Avoid double-barreled questions (questions that ask two things at once) - Objective - Avoid leading questions using emotionally charged words - “Are you in favor of eliminating the wasteful expenses in the budget?” - Instead question be neutral  - “Are you in favor of reducing the budget?” - Reverse the wording of some items - Reversing some wording helps prevent when participants agree or disagree with everything because they are not paying attention - (e.g. don’t use a scale ranging from strongly disagree to strongly agree) - PRACTISE - Define the problems with: - How often do you punish your child? - Curating development and protecting the environment should be a top priority for our city - You wouldn't say that you are in favor of capping tuition rates, would you? - Open-ended versus closed-ended items - Closed-ended: - Give a limited number of responses - Must be designed carefully - Risk of missing information - Easy to score - For a categorical item you provide a list of options - Example: “Which genre of music do you enjoy the most?” - Options: pop, rock, rap, jazz, metal - For continuous we use rating scales - Pick a number on a scale (e.g. 0-10) - Key decisions researchers must consider - How many points on the scale - What are the anchors (labels on the ends) - Likert scales - Common type of rating scale used to assess degree of liking or argument - Usually 7 points but can also be 5 or 9 - Open-ended - Allow participants to respond however they want  - Example: “Which genre of music do you enjoy the most?” - Answer here: _______ - Provides a lot more information - Things to consider: - Effortful for the participants - Need to decide how to score responses (coding scheme) - Scoring the responses takes time - Assembling the Survey - Order matters because questions can bias responses to later questions (item order effects) - Place less sensitive/ objectionable items before more sensitive items - Demographic items are rarely biased and thus go last - Should be organized in a coherent visually pleasing format - Font sufficiently large - Not too many items per page (reduce scrolling) - Viewable on different devices - Samping issues - Population - Entire group we want information about (all potential subjects) - Defined by the research question - E.g. children - E.g. children in kindergarten - E.g. Children in kindergarten in London - Samples - Subgroup of subjects chosen from the population - E.g. children who we studied - E.g. children in kindergarten who we studied - E.g. Children in kindergarten in London who we studied - Generalization and samples - Want to apply results obtained from a sample to the population - Thus the sample must represent the population - Simple random sampling - Everyone in the population has an equal chance of participating  - Sample should look similar to population - E.g. same % of each ethnicity, age, gender, personality traits - Almost impossible to obtain - Stratified random sampling - Important subgroups are identified - E.g. ethnicity, gender, age, income, etc. - Obtain a random sample of each subgroup to mirror the population - E.g. subsample students from public, catholic, and private elementary schools in london - Nonrandom sampling - Everyone in the population does not have an equal chance of participating - May lead to biased sample - As characteristics differ from the model population - Often due to selection bias - Sampling procedures that favor certain characteristics - Convenience sampling - Using participants who are easily available - Very common - Limits external validity - Student populations - 81% of research participants are university students - Advantages - Easy to access to students, free - Educate students about research - Disadvantages - Less variability in age, education, intelligence, wealth - Internet populations - Recent increase in amount of low paying, online research - Tend to - Have a lot of free time - Be lower income - Be more tech savvy - Take research less seriously - Voluntary participation  - Ethically participation must be voluntary but this can affect external validity - Volunteer bias: volunteers are different than non-volunteers - More educated - Higher social class - Higher intelligence - Higher need for approval - More social - More “arousal seeking” - Women volunteer more - Generalizability - Is random sampling always necessary? - Need to consider how much participants characteristics are likely to affect results - Sometimes very much (political polling) - Sometimes less so (vision and reaction time) - Textbook - Cyclical model for scientific research in psychology: - Researchers begin by formulating a research question often inspired by existing research literature, informal observations, or practical problems - They design and conduct an empirical study to address the research question - Researchers then analyze the data collected during the study - Based on the data analysis conclusions are drawn regarding the research question - Finally, the results and conclusions are published, adding to the body of research literature and potentially inspiring new research questions - Example of this model: talkativeness in men vs women - Research stemmed from stereotype and existing claims in literature - A review of the research literature revealed a lack of investigation into the question - They performed a carefully designed empirical study, analyzed data, and found minimal differences in men and women - Then they published these findings potentially inspiring new research  - Example 2 of this model: potential negative effects of cell phone use on driving - Widespread adoption of cell phone in 90s raised concerns about potential effects - Prior research established verbal tasks while performing motor tasks leads to impaired performance - Researched conducted studies to examine impacts - They revealed that cell phones do negatively impact driving as they affect drivers ability to detect hazards, reacts to situations, and maintain vehicle control - They then published their findings which contributed to the growing research on this topic - Then further research explored this issue demonstrating that conversations on phones cause larger risks than in person conversions when driving Interpreting Graphs - Categorical - Categorical data: each value represents a discrete category (order does not matter) - E.g. animals at the zoo, 1 tiger, 2 lions, 3 monkeys - Categorical graphs - Pie charts - Great for displaying relative frequencies (parts of a whole) - Not great for displaying absolute frequencies - Bar graphs - Compares a series of categories by representing each one as a bar - Numerical - Numerical data: each value represents either a real number or a place on a continuum (order matters) - Real number (e.g. age) - Continuum (e.g. a rating scale) - Numerical graphs - Histograms - Shows the distribution (shape) of a numerical variable - Gives you a visual, intuitive sense of the mean, range, skew, and potential outliers  - Scatterplots - Shows how two numerical variables are associated with each other - Works particularly well when both variables are continuous - Line graph - Tells you how two variables are associated by drawing a line through a series of places where X and Y intercept - X-axis: usually a discrete variable - E.g. volume of avocados sold (binned into discrete values) - Y-axis: usually a continuous variable - E.g. price of avocados - Time series graphs (if data has been collected over time) - A special kind of line graph showing how something changes over time - X-axis: Time, usually as a discrete variable - E.g. the years 2015-2018 - Y-axis: a continuous variable you care about - E.g. price of avocados - For line graph or time series graphs - Trickery with the axes - Truncating an axis - Restricting range to maximize differences - E.g. including implausible values - Expanding an axis - Using too broad a range to minimize differences - Ignoring conventions - E.g. values should go from small to large - Comparing non-equivalent data - E.g. two different Y-axises on the same graph - Especially with time series graphs (x-axis) - What time period should you display - E.g not starting at baseline - Types of numerical data - Discrete: the variable has a discrete finite number of values - E.g. day of the month which u bought x - Continuous: the variable has an infinite number of values - E.g. average number of x u buy a month  - Usually we assume they are normally distributed - To make something normally distributed we take log 10 of each value and use that instead (log transforming) - You can also bin it into discrete numerical values - E.g. 1 = 0-99k, 2 = 100k-199k, etc.. - Textbook notes - Graphs allows us to communicate data when precise detail is not require, they also allow us to show a visual trend or comparison where there is a relationship between data points - Effective graphs use appropriate formatting - Title, axis labels, legends, footnotes, an appropriate representation of axes and scale - Title should clearly explain what x and why axes represent and including units of measurement - Legends provide a key to the various data plotted on a graph - Footnotes provide further information - Source data, sample size, any other relevant contextual information - An uncluttered visual style that makes it easy for readers to interpret the data - Graphs should prioritize a clean and easily interpretable visual style, emphasizing data ink over non-data ink - Reducing clutter, highlighting important information, and using colors and patterns thoughtfully - A vertical y axis should start at zero to avoid data distortion - The only exception is when there are negative data values - Two-dimensional graphs are generally preferred over three dimensional graphs as it can distort scare and be misleading - Confidence intervals are used to indicate the range of error in estimates derived from sample data - Typically are displayed as error on bar graphs - The type of graph must be appropriate for the type of data represented - Categorical data - Falls into discrete categories with no intrinsic order should be presented with bar graphs, clustered bar graphs and stacked column charts - Ordinal data (similar to categorical but with a clear ordering of variables)  - Should be represented using bar graphs and histograms - Continuous data (measured on a numeric scale) - Can be presented using line graphs and scatter plots - Interval data (which can have both an order and equal spacing between categories) - Can be presented using bar graphs, histograms, and box plots - On the other hand are kinds of graphs and what to put in them - Bar graphs - Suitable for presenting categorical data - Vertical bar graphs for estimates for 2-7 groups - Horizontal bar graphs for 8 or more groups - Clustered bar graphs - Used to display two or more categories on one graph - 100% stacked column graphs - Compare the percentage contribution of each value to a total across categories - Line graphs - Illustrates trends over time for continuous data grouped into ranges - Histograms - Display the frequency distribution of continuous data grouped into ranges - Scatter plots - Show the relationship between two continuous variables - Box plots - Display the variation in a set of interval data highlighting the minimum, lower quartile, median, upper quartile, and maximum value - Pie graphs - While popular, have limitations and should be used with caution - Must add up to a total of 100% Measurement - Operationalizing your variables - What are we measuring - Variables and particularly constructs - Defining your variables  - Conceptually vs. operationally - Types of measurement - Self report, behavioral, physiological - How do you choose - Previous research, theory, methodological advances, and feasibility - How was this variable measured in previous studies - Part of why doing a literature review is so important - Reliability - Does your measurement consistently measure the same thing - We are measuring variables (things that vary) - E.g., age, gender, shoe size, extraversion, aggression - Variables that cannot be observed directly - E.g. traits, emotions, attitudes, abilities - Relevant constructs - Predictor variables: what they listen to - Let's call it musical exposure - Dependent variable: will affect them - Openness to experience? - Conceptually define your construct - Openness to experience: the personality trait of being intellectually curious, creative, and imaginative - Operationalize your construct - Is to concretely specify how it will be measured or manipulated - Every variable must be operationalized - Openness to experience has an established scale we can use - A person’s mean scores on the 10 openness items from the BFI scale (two-items reverse-scored) - Musical exposure does not have an established measures - So we must produce one ourselves based on our conceptual definition - Options for operationalizing - Amount of exposure  - Number of hours spent listening to music per week - Range of exposure - Number of different musical genres available in household - Method of exposure - Time spent listening to musical recordings vs. live performances - Musical knowledge - Performance on a music recognition quiz - Operationalizing your variables - What are we measuring - Variables and particularly constructs - Defining your variables - Conceptually vs. operationally - Ways to measure your variables - Self-report, behavioral, physiological - Some types of measurement - Self report measures - Interviews or questionnaires - People report their beliefs, behavior, history, etc. - Behavioral measures - Observations of behavior - Could be naturally occurring (e.g. flirting in a bar) - Or lab induced - Physiological measures - Assessment of bodily states - E.g., brain imaging (fMRI, PET); heart rate - Accuracy - No measure is going to be completely accurate - E.g. scale will be slightly off, questionnaire score wont be identical - True score: the “real” score on the variable - Obtained score: the score the measure gives - Measurement error: difference between true score and obtained score - Want to minimize measurement error - Reliability  - Does your measure give consistent results under the same conditions - E.g. if nothing changes - Scales should give the same weight - Questionnaire results shouldn’t change if taken twice - How do you test the reliability of a self-report measure - Test-retest reliability - Same test is given twice with some time in between - Good for stable qualities (e.g. personality), not temporary states (e.g. mood) - Parallel-forms reliability (if applicable) - Different forms of the same test used - Internal consistency - Test it with split-half correlation - Top-half of questionnaire is compared to the bottom half - Test it with cronbach's alpha - Tests how all the items are interconnected - How do you test the reliability of observational measures - Face validity - Does the measure look like it measures the thing its meant to measure - Content validity - Does the measure capture all the important facets of the construct - Criterion validity - Convergent validity - Does it correlate with similar variables - Predictive validity - Does it predict expected outcomes - Discriminant validity - Your measure should NOT correlate with theoretically different variables - Also called divergent validity - If you don't achieve discriminant validity your measure is likely to broad - Example: - Love languages as sample items - It's more meaningful to me when: - Im complimented by my partner on my appearance  - My partner takes the time to listen to m and really understand my feelings - I receive a loving note/text/email for no special reason from my loved one - My partner and I hug - Measuring using a likert scale (1-7, from not true at all - very true) - Textbook chapter 4 - Folk psychology - People have intuitive beliefs about human behavior, thoughts, and feelings, collectively referred to as folk psychology - While some may be accurate much of it is not - Examples: - The idea that releasing anger through actions like punching or screaming reduces anger, when research shows it does not and instead increases anger - The belief that people won't confess to crimes they didn’t commit unless tortured. Yet research shows that false confessions are common and happen for a variety of reasons - People only use 10% of their brain power - Most people have a midlife crisis in their 40s or 50s - Matching teaching style to learning style is optimal for student learning - Low self-esteem is a primary cause of psychological problems - Psychiatric admissions and crime rates rise during full moons - Reasons for inaccurate intuitive beliefs - Limitations in human cognition - Forming accurate beliefs requires strong observation, memory, and analytical skills which humans dont natural possess to the degree needed - Heuristics - People depend on mental shortcuts (heuristics) to form and uphold beliefs - If a belief is widely held, especially by experts, and seems intuitively possible it's often accepted as true - Confirmation bias  - People tend to focus on evidence that confirms already existing beliefs they have and disregard contradictory evidence - For example, if someone believes that women are more talkative than men they may look over quiet women and talkative men, only noticing a remembering loud women and quiet men - Wishful thinking - People may cling to beliefs because they find them appealing even if they aren't true - Like the belief that calorie reducing diets are effective long term for weight loss, which isn't true but we want it to be - Skepticism in science - scientists , particularly psychologists, recognize their own susceptibility to intuitive but inaccurate beliefs to counter this they - Consider alternative explanations - Seek evidence, especially systematically using collected empirical evidence - Evaluate the credibility of claims and sources - Tolerance for uncertainty - Scientists understand that there are many unanswered questions and embrace uncertainty - While uncertainty poses practical challenges it fuels scientific inquiry and offers opportunities for research to provide answers Replicability and Open Science - The open science movement - What creates false positives - Incentives to publish - Academics are rewarded for publishing (with jobs, grants, respect, etc.) which can motivate people to take shortcuts - Questionable research practices - Little ways to adjust your design analysis and reporting so that you can boast the desired effect of P <.05 (P less than 0.05) - P value means:  - The probability that you would get these results if the null hypothesis were true - Assuming your effect doesn’t exist, how likely are these results? - You should get a false positive 1/20 times - What happens if you try the same test thing 20 different times (avoided using QRPs) - Specific questionable research practices - Measure depend variable in multiple ways - Measure and test them all (only reporting the one that works) - E.g., how to measure age - Years, months, mentally - Gradually add more observations - The key analysis after every 10 participants collected, stopped data collection when statistically significant (p