Lecture Statistical Inference MH 2024 PDF

IoPPN 17th October 2024 Dr Moritz Herle Introduction to Statistics SGDP Centre Statistical Inference DD/Month/ Professor/Dr: Topic title: YYYY This lecture Learning objectives: Und...

IoPPN 17th October 2024 Dr Moritz Herle Introduction to Statistics SGDP Centre Statistical Inference DD/Month/ Professor/Dr: Topic title: YYYY This lecture Learning objectives: Understand and define key concepts: - Sampling variation and sampling distribution - The normal distribution - The Central Limit Theorem - Standard error and standard deviation - Confidence intervals - P-values The basics These lectures assume you’re familiar with concepts like means and standard deviations (descriptive statistics), and some basic R functions (from the previous lecture) If you need a refresher, see Chapters 3, 4, and 5 of the Navarro textbook for a gentle introduction: https://learningstatisticswithr.com/ British Social Attitudes Survey – National Centre for Social Research Think - Pair – Share Discuss what you are seeing on the figure with the person next to you. British Social Attitudes Survey – National Centre for Social Research Where do these numbers come from? 33,57 4,780 completed 7 the questionn were eligible aire 36,989 addresses addresses from the Postcode 6,10 Address File from the Post 8 Office agreed to participa te Probability versus Statistics? t i on pul a e po t i r En Think – Pair - Share What issues could there be if you use a subsample to get information about the whole population? Samples & populations A fundamental goal of statistical inference: We want to know about the population… But what we have are samples It’s (usually) impractical or impossible to measure the whole population So we want to make generalisations (inferences) from samples to the population How do we ensure that we do so reliably? Sampling variation The random differences between our sample(s) and the population Sample Between samples there’s a lot of sampling 2 variation: some will be be better or worse estimates of the population, just by chance Sampl (As opposed to sampling bias, which comes Sampl e4 from systematic forces), e.g.: e3 You’ve only sampled people who answer the telephone You’ve only sampled university students Sa mpl e1 How do we get the best estimate of the population values in the face of sampling variation?  Before we start, note that this lecture Frequentist vs. Bayesian (and this course) is from a frequentist statistics perspective The other main “school” of statistical thought, Bayesian statistics, does things differently Both valid, both have pros and cons Frequentist statistics are more widely- used (p-values, significance, null hypotheses, Sir Ronald Fisher confidence intervals) Bayesian statistics becoming more popular (Bayes Factors, priors, credible intervals) Rev. Thomas Bayes For a Bayesian introduction, see Normal distribution aka Gaussian Distribution aka Bell Curve µ = Mean of the distribution = 0 σ = Standard deviation of the distribution = 1 Normal distribution aka Gaussian Distribution aka Bell Curve µ = Mean of the distribution σ = Standard deviation of the distribution µ = Mean of the distribution σ = Standard deviation of the distribution Normal distribution Normal distribution Probability density Population mean = 100 Population mean = 110 The big question in (frequentist) statistics Imagine it looks like there’s an effect in our study E.g. the groups look different (different means) How likely is it that these sample differences would have arisen (through sampling variation) if there’s really no effect in reality? The big question in (frequentist) statistics The “would have arisen” part implies that there might have been a chance for other data to “arise” Frequentist statistics is all about what might happen if we repeated a study/experiment/etc over and over, sampling again and again (frequently) from the same population This is where sampling distributions come in The sampling distribution of the mean Two things to look at: 1. The mean of the sampling distribution 2. The shape of the sampling distribution The mean is near-identical to that of the population! The sample means are (approximately) normally distributed 200 samples, n = 5 200 samples, n = 5 (pop_norm2 (pop_norm1) What are the means and standard deviations of these sample distributions? DD/Month/ Professor/Dr: Topic title: YYYY The standard error The standard deviation of the sampling distribution is called the standard error It gets smaller as the sample size increases It’s a useful measure of sampling error variation (or precision) Standard deviation is a measure of the spread of a distribution: Standard error is a measure of the spread of the sampling error 200 samples, n = 5 200 samples, n = 50 (pop_norm1 (pop_norm1) 200 samples, 200 samples, 200 samples, n=5 n = 50 n = 250 (pop_norm1) (pop_norm1) (pop_norm1) The Central Limit Theorem 1. The mean of the sampling distribution is the same as the mean of the population 2. The standard deviation of the sampling distribution (i.e. the standard error) gets smaller as sample 3. The shape of the sampling distribution becomes normal as the sample size increases Why this is relevant We usually only have one sample in reality And in reality, we don’t know what the population distribution really looks like That’s what we’re trying to find out! But we can use our knowledge of the Central Limit Theorem and standard errors to help us make inferences The first application of this knowledge is in calculating confidence intervals The normal distribution ~95% of the estimates lie within 1.96 standard deviations either side of the mean of a normal distribution The normal distribution again IF 95% of the estimates are within ±1.96 SDs of the mean of a normal distribution… …AND if the sampling distribution is normally distributed… …AND if the mean of the sampling distribution is the same as the population mean… …AND if the standard deviation of the sampling distribution is called the standard error... …THEN in 95% of repeated samples, the observed sample mean lies within 1.96 standard errors of the population mean. Fair enough, but this is backwards: we don’t know the population Flip things around “95% of observed sample means lie within 1.96 standard errors of the population mean” pop_mean – (1.96*SE) ≤ samp_mean ≤ pop_mean + (1.96*SE) Rewrite… samp_mean – (1.96*SE) ≤ pop_mean ≤ samp_mean + (1.96*SE) So… In 95% of repeated samples, the ±1.96SE range will contain the population mean That range is a 95% confidence interval (See Navarro, Learning Statistics With R, p.321, for a slight complication) Back to populations (and sample sizes) Remember our very first population, “pop_norm1”, the normally- distributed one? Let’s try calculating confidence intervals for the different samples from that population Bigger sample size -> smaller confidence interval Answering our question? p-values In frequentist statistics, p-values are a favoured tool to test hypotheses Assume that there’s no real effect - the null hypothesis Make a null population distribution: what would we expect to see were we to take repeated (frequent) samples in a situation where there’s no effect? Is our sample part of this null population distribution? Compare the observed effect to this null distribution to get a p-value The p-value is the probability that the result, or a more extreme result, would be observed if in fact the null hypothesis is true (and where all other assumptions hold) Standard practice (for good or for ill) is to accept results where p < 0.05 as “statistically significant” The alpha level is 0.05 A one-sample example We have a sample of 36 children from a school that’s been randomly assigned a new educational programme Does the programme boost their IQ? In this case, we know the population parameters: IQ tests are standardized so the mean is 100 and the SD is 15 This is a “one-sample” test, since we just have one sample that we’ll compare to known population values What’s the probability that the kids’ average IQ will be ≥ 105? Steps to getting a p-value 1. Calculate the standard error. SE (SD of sampling dist) = SD/sqrt(n) Use the population SD 15/sqrt(36) = 2.5 2. How many standard errors is this sample’s mean from the population mean? Use a z-score to put this in standardized units (mean of zero, standard deviation of 1) Use a z-score: z = (samp_mean – pop_mean)/SE z = (105-100)/SE z = 5/2.5 = 2 3. Work out the area under the curve of the standardised sampling distribution to the right of our z-value – that’s the p-value Standardized Sampling Distribution of Mean IQ z-score table tells us that this area is equal to 0.023 (that is, the probability of observing an n=36 sample with a mean IQ of 105 is 2.3%) Our p-value is 0.023 Source p-values and statistical significance In the above case, if our alpha level was the standard 0.05, we would conclude that the sample was rare enough to be statistically significantly different from the rest of the population Note that this is different from saying it’s rare to have an IQ above 105: we’re talking about a sample average Note also: if we’d collected a smaller sample, the standard error would be wider, and we might not be able to conclude that the average of 105 was anything special SE (SD of sampling dist) = SD/sqrt(n) Summary We want to know about the population by taking samples Sampling variation is ever-present; larger numbers help deal with it to a degree Frequentist statistics rely on the central limit theorem: that sampling distributions are normally distributed Given the central limit theorem, we can calculate confidence intervals and p-values p-values (and confidence intervals!), which will appear many times in future lectures Read on: Chapter 10 and Chapter 11.1 to 11.7 in Navarro (2018) https://learningstatisticswithr.com/lsr-0.6.pdf DD/Month/ Professor/Dr: Topic title: YYYY Additional points: What if the outcome variable is not normally distributed? Poisson distribution, often used for count data. Examples: Number of football goals scored in a game, number of stressful live events in childhood… DD/Month/ Professor/Dr: Topic title: YYYY From the poisson population… 200 samples, n = 5 200 samples, n = 50 DD/Month/ Professor/Dr: Topic title: YYYY 500 samples, n = 200 DD/Month/ YYYY Professor/Dr: A normal curve again, even from a very differen Topic title: