Lecture 4

Estimation

• Estimation is the process of approximating a population parameter's value. It involves:
• Conducting experiments.
• Sampling populations.
• Examining associated data.
• Accuracy and precision are central to estimation, and intertwined ideas.
• Increasing precision can reduce accuracy, and finding the right balance improves outcome estimates.
• Benefits include reduced noise/unexpected variance and more meaningful results.

Accuracy and Precision

• Accuracy and precision are intertwined, but there is a trade-off where more precise estimates become less accurate.
• Selecting the right level enables improved estimates of outcomes, reducing noise and variance, as well as becoming more meaningful.

Parameters

• Population parameters are theoretical, denoted as μ (mu) and σ (sigma).
• Sample parameters are observed, denoted as x̄ and s.

Generalization and Sampling

• A sample representative of the population allows reasonable estimation of population parameters.
• A convenience or "ad-hoc" sample involves sampling those easily accessible, though this method can be biased. For example:
• Sampling squirrels in Ontario versus only those at UWO.

Random Sampling

• Every entity in a sample has an equal chance of being selected.
• Random sampling reduces bias, but achieving a truly random sample can be challenging.

Sampling Error

• Sampling error is the difference between a population parameter and a sample statistic.
• It occurs due to the random selection of specific population members.
• If the sample is random, this error is unbiased, with no participant characteristics affecting sampling likelihood.
• Error occurs due to the differences in who was and was not sampled.

Sampling Bias

• Sampling bias can take the form of:
• Non-responders not wanting to participate.
• This will lead to bias if a systematic trait motivates who responds.
• Self-selection, addressed by advertising studies effectively to influence who responds.
• Use of convenience samples, commonly used in psychology, requires careful generalization.

Sample Representativeness

• Sample representativeness is a major factor for accurate population estimate.
• If a sample is not representative, conclusions may be invalid and biased toward the sampled characteristics.
• Estimates can be biased even with random sampling.

Measurement Precision

• The design of your measurement instrument is critical when making population inferences.
• Measurements must be precise enough without adding excess noise.
• Conclusions will not generalize beyond the lab if measurements are not carefully selected.

Non-Sampling Errors

• Measurement error includes:
• Lack of validity or reliability of measurements
• Inaccurate recording of responses.
• Errors can also be related to:
• Calculation or mathematical mistakes in analysis.
• Data summarized inaccurately.
• Errors in statistical analysis (e.g., wrong analysis).
• Misinterpretation of data results in:
• Statistical tests not being accurately interpreted.
• Overstepping methods.

Point Estimates

• A single value (statistic) that best represents population value
• Is often the sample mean (x̄), but could also be the median or mode.
• Selection among these options should take into consideration the following:
• An estimator that is unbiased will target the population parameter, and sometimes be a bit higher or lower, but it will work out similarly on average.
• A biased estimator will likely overestimate or underestimate it where there is a systematic deviation between the estimator and the parameter.
• Mean is an unbiased estimator because it overshoots and undershoots μ equally.
• Standard deviation is biased as it is more likely to undershoot σ.

Biased Estimator

• To correct bias when computing a sample's standard deviation, use Bessel's correction.
• This entails dividing by n-1 instead of n in the denominator of the formula.

Efficiency

• Efficiency is the sampling variability of an estimator.
• In skewed/non-normal data, the median may be better than the mean due to its lower sensitivity to outliers.
• If two unbiased estimators are compared (e.g., sample mean and median), the one with the smaller variance is more efficient given sample size.

Problems with Point Estimates

• Estimates are made from single samples and commonly assumed to be precise.
• However, single samples almost never exactly represent the true population parameter.
• Random sampling leads to variability in sample makeup, so the estimator varies from the quantity it should estimate. In reality:
• Estimates can vary substantially from one sample to the next.

Interval Estimates

• Because point estimators vary, a range of values likely to contain an estimator can be more reliable.
• The more confident one is, confidence in the estimation process increases.
• The estimation process starts before data collection.
• Using an interval makes it likelier to capture the parameter.
• Determined by the researcher in adcvance:
• Interval size depends on desired confidence level.
• Narrower ranges (for the same confidence level) indicate greater precision than wider ranges.
• To find the upper and lower boundaries of intervals, this will contain the population parameter.

Confidence Intervals

• The distance between these boundaries indicates confidence of how they describe the estimation process.
• Confidence intervals use a range of values within which there is confidence that μ lies. These:
• Offer a probabilistic statement about the interval within which μ happens to be.
• The more accurate the sampling plan/research design, the more likely the CI includes μ.
• Certainty means determining how certain the specified range of values contains μ:
• It depends on what you think you can get out of distribution.
• Are most frequently calculated using a 95% confidence interval. Reported as '95%CI' or 'CI95':
• This is a 95% confidence that what you expect is within the calculated interval.
• With a 5% margin of error.

Calculating Confidence Interval

• Done by finding the:
• Mean of the estimator, +/- the variation in the estimate.
• Confidence level (e.g., 50%, 95%), often denoted as 1 - α.
• Which relies on the theoretical distribution of the parameter (e.g., μ).
• Increasing the width of the interval boosts confidence levels because if the interval is too wide, it becomes uninformative.

Inferring the Population

• To make inferences about the population, one sample is collected in a single experiment.

• There are two methods to get the population parameter estimate:

• Calculated using a formula.

• Bootstrap resampling.

Confidence Interval Formula

• Confidence interval for mean of normally distributed population: CI = μ + Z* (σ /√N)

  • CI = Confidence Interval
  • μ = Population Mean
  • Z* = Critical value of the Z Distribution
  • σ = Population Standard Deviation
  • N = Population

• When calculating a CI using a formula, one must make certain assumptions about the sample drawn. For the CI to be meaningful the:

• Sample will truly be a random representative of the population. However:

  • In Psych, convenience samples tend to be used instead.

• Sample will be normally distributed.

  • Though that will be very unlikely in real life.

• Scores will be independent.

  • Meaning there is no relationship between collected scores.

Bootstrapping: The Logic

• If the population data is available, the mean could be calculated:
• Which would provide a perfectly accurate estimate of the true population mean, if sample is large enough.
• Bootstrapping should be used if this is not possible because:
• It is too costly.
• or, Too time consuming.
• We must instead, build a model of the population and sample from that.

Bootstrap Method

1. Use the bootstrap method to resample your data and build a DOSM by:
   • Using the sample available, and make for lots of iterations.
   • Resample the sample by selecting samples of size N with replacement.
   • Record the sample statistic (e.g., the mean) into an array, and,
   • Return the array/distribution of sample statistics.
2. Then determine the confidence interval based on the DOSM we obtained by:
   • Comparing this to the calculated confidence interval, and:
   • Comparing your estimator to the true population mean.

Bootstrapping Magic

• This process allows for approximation of the population value by determining the interval which that value is likely to fall, if you meet the conditions that you have selected a representative sample.
• Other benefits include:
• Being a universal technique applicable to any statistic (not just the mean).
• Not requiring assumptions about the underlying distribution.
• Disadvantages involve:
• Requires programming skill.
• Requires computing time.
• Can be noisy is sample has a small N.

Confidence Intervals Notes

• If you calculate the 95% CI for your mean:
• YOU CAN be 95% confident that the interval contains the parameter, where CI tells you what range of values you can expect to find if you re-do the experiment in exactly the same way.
• YOU CANNOT be 95% certain that the parameter is in the interval.
• It is the boundaries of the interval that vary, and not the population parameter.
• The confidence interval will not be able to tell of you found the true value: -- It is instead based on the likelihood the result is sampled, not on 100% truth of you found the population.

Confidence Intervals

• By estimating where the interval within which the population parameter of interest lies, one can:
• Is reliant on solid methodologies.
• Sampling techniques
• Measurement calibration.

Effect Size or Magnitude

• Often, we want to know:

• If one group of people differs from another.

• If a treatment was effective.

• Or if there is a relationship between two variables.

• The Effect size is a quantitative measure of any association/experimental effect:

• So we use intervals to help answer the question(s).

• Does the confidence interval around the mean of group 1, include the mean of group 2?

• If there is an overlap between the confidence intervals, then, there likely is no statistically significant difference between what you measured and your estimate of the population.

• But of course the confidence interval says nothing about "truth", of the population - only your likelihood of sampling and measuring it.

• If there is no overlap between the confidence intervals, then, there likely IS a statistically significant difference between what you measured and your estimate of the population.

• Once again, that this does not measure "truth", but rather the likelihood.

• How strong is the relationship or association between variable x and variable y?

Explore estimation methods for population parameters through experiments and sampling. Learn how accuracy and precision are related in estimation. Understand the importance of balancing accuracy and precision to improve outcome estimates and reduce variance.