Part 2 PDF - Hypothesis Testing

In blue or highlighted: what was said or emphasized by the lecturers Chapter 8 – Hypothesis testing: significance, effect size and power Inferential statistics and hypothesis testing Inferential statistics allows us to observe samples to learn more about the behavior in population...

In blue or highlighted: what was said or emphasized by the lecturers Chapter 8 – Hypothesis testing: significance, effect size and power Inferential statistics and hypothesis testing Inferential statistics allows us to observe samples to learn more about the behavior in populations that are often too large or inaccessible to observe We use samples because we know how they are related to populations. The sample mean in an unbiased estimator of the population mean. On the basis of the central limit theorem, we know that the probability of selecting any sample mean value from the same population is normally distributed We expect the sample mean to be equal to the population mean Principles of hypothesis testing Significance testing = We test a hypothesis about a parameter of a population (mean), using sample data measured in a sample. Hypothesis are therefore always about populations, never about samples. Test of significance are used to determine the likelihood that a hypothesis about a population parameter (mean) is true The null hypothesis (H0) = hypothesis of equality; is a statement about a population parameter, such as the population mean, that is assumed to be true o We always start by formulating a null hypothesis about a population parameter (the mean) o Very often, the H0 is not what the researcher expects: the only reason we are testing the H0 is because we think it is wrong o Typically, we therefore try to reject the H0 o The opposing hypothesis is the H0 is called the alternative hypothesis (H1), also called the research hypothesis = is a statement that directly contradicts a null hypothesis by stating that the actual value of a population parameter is less than, greater than, or not equal to the value stated in the null hypothesis o We can never prove our alternative/research hypothesis (H1) about the population directly. We can only show that the H0 is unlikely to be true o In significance testing, we first assume H0 to be true and then see how (un)likely it is, given the sample data that we just obtained o If H0 turns out to be very unlikely, we reject it, and therefore accept H1 How unlikely is unlikely enough to reject H0? What threshold to use? o We call this threshold the “level of significance” (alpha) = based on the probability of obtaining a statistic measured in a sample if the value stated in the H0 were true o In social sciences, we typically use a 0.05 level of significance: alpha= 5% à because regardless of the distribution in a given population, the sampling distribution of the sample mean is approximately normal. Hence, the probabilities of all other possible sample means we could select are normally distributed o Meaning: we make sure that the probability of falsely rejecting a true null hypothesis (type 1 error) is not greater than 5%; Or the chances of the null hypothesis being true, despite the data we have, is no greater than 5%  In blue or highlighted: what was said or emphasized by the lecturers Level of significance: Type 1 error = Probability that you are going to make a mistake and accept H1 when you should have accepted H0 Type 2 error = is the probability of retaining a null hypothesis that is actually false Type 3 error = when we fail to reject a H0 because we placed the rejection region in the wrong tail Power in hypothesis testing = the probability of rejecting a false null hypothesis. Specifically, it is the probability that a randomly selected sample will show the null hypothesis is false when the null hypothesis is indeed false 4 steps in significant testing (when done by hand) State H0 and H1 Set the level of significance/alpha = 5% Compute the test statistic o One-independent sample z-test (−" R]^_`a)bZ = K √2 o One-independent sample t-test (−" =]^_`a)bZ = ˆK √2 Make a decision to reject/accept H0 (decisions are about H0) o Compare the calculated test statistic (obtained value) with a critical value found in a table § Critical value = the cutoff value that defines the boundaries beyond which less than 5% of sample means can be obtained if the null hypothesis is true. Sample means obtained beyond a critical value will result in a decision to reject the null hypothesis § Obtained value = the value of a test statistic. When obtained value > critical value, we reject H0 § Z-critical (rejection region) = 1.96 for two-tailed H1 (alpha=2,5%) § Z-critical (rejection region) = 1.65 for one-tailed H1 (alpha = 5%)  In blue or highlighted: what was said or emphasized by the lecturers o Rejection region = region beyond a critical value in a hypothesis testing. When the value of a test statistic is in the rejection region, we decide to reject the null hypothesis; otherwise, we retain the H0 State your conclusion referring to the population Population Sample Distribution of distribution distribution sample means What is it? Scores of all Scores of a select All possible sample persons in a portion of persons means that can be population from the selected, given a population certain sample size Is it accessible? Typically no Yes Yes What is the shape? Could be any shape Could be any shape Normal distribution Idea behind the one-independent sample z-test We use z-test when the standard deviation is known Independent = when sampling data, each case is independent of the other, no direct affect between data; no causality between one person to the other Non-directional à 2 rejection regions (2,5% level of significance each) If the z value is higher, it is more difficult to reject H0 à we retain H0  In blue or highlighted: what was said or emphasized by the lecturers Chapter 9 - Testing means: one-sample and two-independent-sample t Tests One-independent sample t-test Three assumptions for a one-sample t test 1. Normality: data in the population being sampled are normally distributed. In larger samples (n>30), SE is smaller and this assumption becomes less critical as a result 2. Random sampling 3. Independence: one outcome does not influence another. Using random sampling usually satisfies this assumption We use t-test when the standard deviation is unknown (−" =]^_`a)bZ = ˆK √2 ^ means estimate of SD As the estimate of sigma, we use the sample SD o Estimated standard error = estimate of SD of a sampling distribution of sample means selected from a population with an unknown variance. it is an estimate of the SE or SD that sample means deviate from the value of the population mean stated in the null hypothesis Σ(x − M)/ ˆK = N 2−1 Follow the same steps as for z-test Know the “degrees of freedom” (df) o For the one-sample t-test: df = n-1  In blue or highlighted: what was said or emphasized by the lecturers o As sample size increases, df also increase If obtained value > critical value, we reject H0 The larger the sample size, the more closely a t distribution estimates a normal distribution o There is a greater probability of obtaining sample means that are farther from the value stated in H0 in small samples. As sample size increases, obtaining sample means that are farther from the value stated in the null hypothesis becomes less likely. The result is that critical values get smaller as sample size increases T-critical (rejection region) = 1.96 for two-tailed H1 (alpha = 2,5%) T-critical (rejection region) = (±)1.65 for one-tailed H1 (alpha = 5%) t-test in SPSS The output will not tell us directly whether or not to reject H0 SPSS calculates a p-value = the probability of obtaining a difference (between the sample mean and the population value we tested it against) at least as large as the one that was obtained under the assumption that H0 is true; p- value is the probability of falsely rejecting H0 à type 1 error If p-value ≥ alpha (0.05) we retain H0 If p-value < alpha (0.05), we reject H0 and retain H1 When p is low, H0 has to go SPSS always assumes a non-direction H1 and thus gives us the p-value for an assumed two-tailed rejection region. If our H1 is direction, we divide the p- value by two before comparing it to our alpha The p-value (Sig.) is the region past the t obtained into the tail. As t gets larger, the region gets smaller and so p-value becomes smaller The same t-value (obtained) for a two-tailed H1 (non-directional) is more likely to reject the H0 under a directional H1 (assuming M is in the correct direction) Two-independent-sample t-test Independent samples = are the selection of participants, such that different participants are observed one time in each sample or group Two-independent-sample t test = is a statistical procedure used to compare the mean difference between two independent groups. This test is specifically used to test hypotheses concerning the difference between two population means, where the variance in one or both populations in unknown In terms of the null hypothesis, we state the mean difference that we expect in the population and compare it to the difference we observe between the two sample means in our sample o "C = "/ (;Y "C − "/ = 0) H1 o "C ≠ "/ ;Y "C >< "/ Four assumptions: 1. Normality: data in the population being sampled are normally distributed. In larger samples (n>30), SE is smaller and this assumption becomes less critical as a result 2. Random sampling  In blue or highlighted: what was said or emphasized by the lecturers 3. Independence: one outcome does not influence another. Using random sampling usually satisfies this assumption 4. Equal variance: the variances in each population are equal to each other. This assumption is usually satisfied when the larger sample variance is not greater than 2 times the smaller df = sum of the dfs = df 1 + df 2 estimated standard error for the difference = estimate of the SD of a sampling distribution of mean differences between two sample means. it is an estimate of the SE or SD that mean differences can be expected to deviate from the mean difference stated in the null hypothesis. The higher the SE, the more likely it to retain H0, because more difficult it is to generalize it. à that’s all we need to know. We don’t need the formulas below for now.