Samples, Populations, and Inferential Statistics

• The lecture covers samples and populations, statistical significance, one-tailed and two-tailed tests, and Type I and II errors.

Recap on Correlation

• The correlation of a sample is represented by Pearson's r
• The range of possible values for a correlation is between -1 to +1
• A positive correlation (+) indicates that as scores on one variable increase/decrease, so too do scores on another variable
• A negative correlation (-) indicates that as scores on one variable increase, scores on another variable decrease, and vice versa
• A correlation coefficient can either be weak, moderate, or strong
• Weak correlation: r is between 0 and ± 0.29
• Moderate correlation: r is between ± 0.3 and ± 0.59
• Strong correlation: r is between ± 0.6 and ± 1.00
• Example: r = -0.4 is stronger than r = +0.3

Inferential Statistics

• Inferential statistics are used to make general statements or draw conclusions that apply beyond the sample
• This involves drawing inferences about all scores in the population from just a sample of those scores

Samples & Populations

• A sample is a small number of scores selected from the entirety of scores
• A population is the entire set of scores
• A sample is a subset taken from the full set or population of scores
• Both population and sample refer to scores on a variable
• The population of scores can sometimes be measured
• More often, the population of scores is infinite and cannot feasibly be measured
• The mean of the sample can be used as an estimation of the mean of the populaion

Random Samples

• In statistical inference, it is generally assumed that samples are drawn at random from the population
• Obtaining a random sample of scores entails selecting scores in such a way that each score in the population has an equal chance of being selected
• Manual random number tables, electronic random number generators, and pulling names out of a hat are ways to obtain a random sample
• Most random samples have a mean that is very close to the population mean
• A sample mean obtained from random sampling that is very different from the population mean is relatively rare

Statistical Significance

• Psychologists are interested in which sample means are very unlikely to occur through random sampling
• The extreme 5% of these samples is of interest, and so these samples are called significant
• Significance means that the means of the sample are very different from those of the population from which it was drawn
• Significance at the 5% level means that the sample score lies within the 5% of samples which are most different from the population
• This 5% is obtained from looking at the extreme lower 2.5%, and the extreme upper 2.5%
• Scores in the middle 95% of scores are likely, and scores in the extreme 5% (extreme upper and extreme lower 5%) are unlikely

Null Hypothesis

• The null hypothesis (H0) always makes a statement of no (null) difference/relationship between the values of a population (e.g. means) or between two variables
• H0 = there is no relationship between the two variables being measured
• The null hypothesis is used to define a population in which there is no relationship between two variables (i.e., the middle 95%)
• Whether or not it is possible that the sample comes from this population is determined by the null hypothesis
• If it is unlikely that the sample comes from the middle 95%, the possibility that the null hypothesis is true is rejected

P-Values

• A p-value is used to find out the probability of a result occurring assuming that the null hypothesis is true
• The null hypothesis states that there is no relationship between the two variables being studied (one variable does not affect the other) and that the results are due to chance and are not statistically significant
• An alternative hypothesis posits that the independent variable did affect the dependent variable, and the results are significant in terms of supporting the theory being investigated and not due to chance
• A p-value, or probability value, is a number describing how likely it is that data would have occurred by random chance (i.e. that the null hypothesis is true)
• The level of statistical significance is often expressed as a p-value between 0 and 1
• The smaller the p-value, the stronger the evidence that the null hypothesis should be rejected
• A p-value less than 0.05 (p < .05) is statistically significant and indicates strong evidence against the null hypothesis
• There is less than a 5% probability the null is correct, thus the null hypothesis is rejected
• A p-value higher than 0.05 (p > 0.05) is not statistically significant and indicates strong evidence for the null hypothesis
• The null hypothesis is not accepted, but the rejection of it can be failed

Notes on Statistically Significant Results

• A statistically significant result cannot prove that a research hypothesis is correct (as this implies 100% certainty)
• Results may state that they "provide support for" or "give evidence for" the research hypothesis, as there is still a slight probability that the results occurred by chance and the null hypothesis was correct
• A statistically significant relationship is one that is unlikely to have occurred in the sample if there's no relationship in the population
• The issue of whether a result is unlikely to happen by chance is an important one in establishing cause-and-effect relationships
• A correlation can be weak but still statistically significant
• The association is small, but not zero

Reporting P-Values

• Report exact p values (e.g., p = .031) to three decimal places
• Report p values less than .001 as p < .001
• Use italics for p
• The opposite of statistically significant is "not statistically significant", not "insignificant"

Type 1 and Type II Errors

• Type I error: Deciding that the null hypothesis is false when it is actually true (i.e., a false positive)
• Type II error: Deciding that the null hypothesis is true when it is actually false (i.e., a false negative)
• In psychology, we use a very narrow threshold for statistical significance (p < .05) to reduce the likelihood that Type I or Type II errors are made

One-Tailed and Two-Tailed Significance Testing

• A one-tailed test specifies the direction of the hypothesis
• Example: 'Those who attend 80% of PS219 lectures throughout term will perform better than those who do not.'
• Two-tailed tests allow the hypothesis to go in either direction
• Example: 'There will be a difference in end of term grades between those who attend 80% of PS219 lectures throughout term than those who do not.'
• Two-tailed tests are more widely used

Problems with P-Values

• Problem #1: P Hacking: the inappropriate manipulation of data to produce a statistically significant result
  • Making up data points
  • Removing data points
  • Altering existing data points
  • Running lots of statistical tests until one is found that produces a statistically significant result
• Problem #2: P-values don't show absolute certainty
  • A statistically significant result cannot prove that a research hypothesis is correct (as this implies 100% certainty)
  • P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone
• Problem #3: p < .05 does not imply a strong effect
  • A p-value is a number describing how likely it is that data would have occurred by random chance (i.e. that the null hypothesis is true)
  • It does not measure how big the association or the difference is
  • To investigate the strength of a difference or association, effect sizes must be looked at