Estimation: Population Parameter

Estimation (Chapter 7)

• Focuses on estimating a population parameter, denoted as $\theta$.
• Examples of population parameters include $\mu$ (mean), $\sigma^2$ (variance), and $p$ (proportion).
• Involves point estimation, which uses a single number as the "best guess" for the parameter's value, derived from a sample.
• A point estimator is a statistic used to estimate a population parameter.
• Common estimators include $\bar{x}$ for $\mu$, $S^2$ for $\sigma^2$, $\hat{p} = x/n$ for $p$, and $\bar{x}_1 - \bar{x}_2$ for $\mu_1 - \mu_2$.

Properties of Estimators

• Unbiasedness is a key property, where $E(\hat{\theta}) = \theta$, meaning the estimator gives the correct value on average.
• Bias is defined as $Bias(\hat{\theta}) = E(\hat{\theta}) - \theta$, indicating the deviation from the true parameter value.
• Variance is considered, with preference for estimators with smaller variance for greater consistency.
• Mean Square Error (MSE) is defined as $MSE(\hat{\theta}) = E[(\hat{\theta} - \theta)^2]$, aiming for a small MSE.
• MSE is the sum of the estimator's variance and the square of its bias: $MSE(\hat{\theta}) = Var(\hat{\theta}) + [Bias(\hat{\theta})]^2$

Confidence Intervals (Chapter 8)

• A confidence interval estimates that the interval contains the true value of a parameter.
• For example, there could be 95% confidence that $\mu$ falls between 10 and 12.
• The confidence level is the probability that the interval includes the true parameter value.
• Constructing a confidence interval for $\mu$ often assumes a normally distributed population.
• If $X_1,..., X_n$ are normally distributed with mean $\mu$ and variance $\sigma^2$, then $\bar{X}$ is normally distributed with mean $\mu$ and variance $\sigma^2/n$.
• Standardizing results in $Z = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}} \sim N(0,1)$.
• $P(-z_{\alpha/2} < Z < z_{\alpha/2}) = 1 - \alpha$ and $P(-z_{\alpha/2} < \frac{\bar{X} - \mu}{\sigma / \sqrt{n}} < z_{\alpha/2}) = 1 - \alpha$.
• $P(\bar{X} - z_{\alpha/2} \frac{\sigma}{\sqrt{n}} < \mu < \bar{X} + z_{\alpha/2} \frac{\sigma}{\sqrt{n}}) = 1 - \alpha$.
• The confidence interval is $(\bar{X} - z_{\alpha/2} \frac{\sigma}{\sqrt{n}}, \bar{X} + z_{\alpha/2} \frac{\sigma}{\sqrt{n}})$ for a $100(1 - \alpha)%$ confidence interval for $\mu$.
• $z_{\alpha/2}$ represents the z-score with area $\alpha/2$ to the right.
• When $\sigma$ is unknown, replacing it with $S$ requires using a $t$-distribution: $t = \frac{\bar{X} - \mu}{S / \sqrt{n}} \sim t_{n-1}$.
• Resulting confidence interval: $(\bar{X} - t_{\alpha/2, n-1} \frac{S}{\sqrt{n}}, \bar{X} + t_{\alpha/2, n-1} \frac{S}{\sqrt{n}})$.