Research Methods 1 Lecture 2 Significance Testing PDF

Research Methods 1 Lecture 2 Significance Testing This Session Sampling from a Population Significance Testing A problem of wording Probability and Alpha Levels A worked example Q&A Target Population We ask questions about ALL people in a population ALL depressed...

Research Methods 1 Lecture 2 Significance Testing This Session Sampling from a Population Significance Testing A problem of wording Probability and Alpha Levels A worked example Q&A Target Population We ask questions about ALL people in a population ALL depressed people ALL females ALL English speakers ALL infants ALL people Etc In an ideal world, we would measure them all… Sampling from a Population It is not usually possible to test everyone in a target population Time consuming Expensive Geographically challenging etc Instead, we take a sample from the population of interest Sampling Error We want to know the mean of the population The mean of our sample may be slightly different Sampling Error Example The Population IQ is 100 We take 5 samples from the population: 95 111 88 82 115 Sampling Error Example The Population IQ is 100 We take 5 samples from the population: 95 111 88 82 115 Sample means can be higher or lower than the population mean Sample Mean – Population Mean= Sampling Error Sampling Error Example The Population IQ is 100 Sample Mean – Population Mean = Sample Error 95-100 = -5 111-100 = 11 88-100 = -12 82-100 = -18 115-100= 15 Multiple Samples Taking an average across multiple samples lowers the sampling error The mean of the sample means is closer to the population mean Small Sample Sizes In smaller samples it is likely that Most participants score above the population mean or Most participants score below the population mean Sampling error is higher Sample means are varied Large Sample Sizes In larger samples it is likely that Some participants score above Some participants score below Sampling error is lower Sample means are relatively stable Larger samples are better  Better estimate of the population mean The Sampling Distribution Underestimates Overestimates The Sampling Distribution The sampling distribution of the mean is a normal distribution Underestimates Overestimates ‘Significant’ – A problem of wording This word isn’t specific to statistics ‘My confidence has improved significantly’ ‘That lecture was significantly worse than last week’ ‘The dog ate a significant chunk of my dinner when it fell on the floor’ Significant – noteworthy, remarkable, great, important, meaningful ‘Significant’ in Statistical Terms From now on you should only use the word significant if you are referring to statistical significance E.g. Grades were significantly higher for 1pm classes than for 9am classes. There was no significant difference between grades for 1pm and 4pm classes. Hypothesis Testing In experiments we hope to show that our IV affects the DV This is our Experimental Hypothesis The Null Hypothesis always states that there will be no effect of the IV Hypothesis Testing Inferential statistics are all about the probability that you would find your results if the Null Hypothesis is true in reality (Type 1 Error) In other words: If in reality your IV has NO EFFECT on your DV What are the chances you would have got the same results  The probability that your result is due to chance Probability (p) values The probability of getting your results by chance is represented by a p value p values are expressed as a decimal between 0 and 1 It may help you to think of these as percentages p= 0.50  50/100 or 50% chance of finding the same result p=0.05  5/100 or 5% chance of finding the same result p Values and Significance A p value is almost never going to be zero. The probability of getting your result by chance may be slim but we need to set a threshold for significance In Psychology that threshold is 5% or p = 0.05 This is known as the alpha level Alpha is different in different areas of science Alpha Level = 0.05 An alpha level of 0.05 means there is just a 5% probability that your result is due to chance I.e. if there is no effect of your IV on your DV and you tested your experiment 100 times, you would only find your results 5 times. A low probability of a chance result means your IV probably did affect the DV  It is likely that your result is NOT due to chance  p.050  No significant Difference Summary Only use the word ‘significant’ if you mean statistical significance Alpha in psychology is 0.05 or 5% p values tell you the probability of getting your result or larger by chance p.05 is non-significant Questions?