Statistical Hypothesis Testing and the P-value - PDF

Statistical hypothesis testing and the p-value Avgis Hadjipapas Professor for Neuroscience and Research Methods [email protected] Session LOBs LOB21: Formulate the null and alternative hypothesis for given research scenarios LOB22: Interpret a 95% Confidence Interval and use it to infer statis...

Statistical hypothesis testing and the p-value Avgis Hadjipapas Professor for Neuroscience and Research Methods [email protected] Session LOBs LOB21: Formulate the null and alternative hypothesis for given research scenarios LOB22: Interpret a 95% Confidence Interval and use it to infer statistical significance LOB23: Interpret a p-value and use it to infer statistical significance Statistical hypothesis testing: rationale When we are assessing associations between variables, we are basically assessing two things: the presence of an association and the magnitude of this association Although both are important, the former is somewhat more so.. In other words, in research, it is extremely important to be able to state with high certainty whether an association exists or not in the source population, based on sample estimates Formulating Hypotheses The Null and Alternative hypothesis: intro For any given association between two variables there are 2 possibilities: 1. The association does not exist in the population (i.e. the two variables are not linked) 2. The association exists in the population (the two variables are linked) The Null hypothesis (H0) always states that there is no association between the two variables in the population The Alternative hypothesis (HA) always states that there is an association between the two variables in the population The Null and Alternative hypothesis: intro 2 When we conduct a research study, we first formulate these two hypotheses and then use statistical analysis to decide whether we have enough evidence to reject H0 Remember we are testing a hypothesis about the population BUT (almost) all research takes place in samples! Thus, the use of statistics is essential in order to quantify the inherent error present in our sample estimate (the random error) Hypothesis testing: the basic logic, part 1 Hypothesis testing: a formal process Define statistical null ( H0 ) and alternative hypotheses (HA ) Start by assuming NO association exists in population➔ i.e. start with H0 Define what is sufficient evidence against H0: the significance level Collect some sample data from population (evidence) … Hypothesis testing: the basic logic, part 2 … Does sample estimate provide sufficient evidence against H0 (i.e. no association)? Or alternatively could sample estimate be explained by random error alone, i.e. consistent with expected sampling variation if no association exists in the population Calculate value of test statistic (using sample) Using test statistic derive probability that quantifies our belief against H0 : p-value Interpret p-value: often in the context of the significance level Statistical significance, the p-value and hypothesis testing The p-value The p-value is defined as per below: What is the probability of obtaining an association as strong (or stronger) as the one observed in our sample, if in fact there is no association present in the source population (i.e. H0 is true)? The lower the p-value , the lesser the chance we could have obtained an association this strong (or stronger) in our sample if no true association existed (in the population). Thus , the lower the p-value, the more we think about rejecting H0 (no association exists in population) in favour of HA (association exists in population) Generally, it is true that the stronger the association ( and the larger the value of the test statistic), the lower the p-value. The significance level … or how strong the evidence (estimate of association in sample) should be and how low the p-value should be … Often a binary cut-off is used to say what is sufficient evidence, or how low a p-value should be Often a significance level of 5% is chosen and therefore a p-value of