Statistical Inference: Point Estimation

• Statistical inference involves drawing conclusions about a population based on sample data.

Point Estimation

• Estimators are rules or formulas used to calculate point estimates from sample data.

Desirable Properties of Estimators

• Unbiasedness means $E(\hat{\theta}) = \theta$, indicating the estimator's expected value equals the true parameter.
• Efficiency means $\hat{\theta}_1$ is more efficient than $\hat{\theta}_2$ if $Var(\hat{\theta}_1) < Var(\hat{\theta}_2)$, showing lower variance.
• Consistency means $\hat{\theta}$ converges to $\theta$ as $n \rightarrow \infty$, showing convergence to the true parameter as sample size increases.

Methods of Finding Estimators

• Method of Moments equates sample moments to population moments to solve for parameters.
• Maximum Likelihood Estimation (MLE) selects parameter values that maximize the likelihood function.
• The likelihood function is $L(\theta|x) = f(x|\theta)$.
• The log-likelihood function is $l(\theta|x) = log(L(\theta|x))$.

Interval Estimation

• Interval estimation provides a range of values likely to contain the true population parameter value.

Confidence Interval

• A $100(1 - \alpha)%$ confidence interval for a parameter $\theta$ is an interval $(L, U)$ such that $P(L \le \theta \le U) = 1 - \alpha$.

Common Confidence Intervals

• For the mean $\mu$ of a normal population with known variance $\sigma^2$, the confidence interval is $\bar{x} \pm z_{\alpha/2} \frac{\sigma}{\sqrt{n}}$.
• For the mean $\mu$ of a normal population with unknown variance $\sigma^2$, the confidence interval is $\bar{x} \pm t_{\alpha/2, n-1} \frac{s}{\sqrt{n}}$.
• For the variance $\sigma^2$ of a normal population, the confidence interval is $\left( \frac{(n-1)s^2}{\chi^2_{\alpha/2, n-1}}, \frac{(n-1)s^2}{\chi^2_{1-\alpha/2, n-1}} \right)$.
• For the proportion $p$ of a binomial population, the confidence interval is $\hat{p} \pm z_{\alpha/2} \sqrt{\frac{\hat{p}(1 - \hat{p})}{n}}$.

Hypothesis Testing

• Hypothesis testing is the process of making decisions about population parameters based on sample evidence.

Basic Concepts

• The null hypothesis ($H_0$) is a statement about a population parameter to be tested
• The alternative hypothesis ($H_1$) contradicts the null hypothesis.
• A test statistic is derived from sample data to decide whether to reject the null hypothesis.
• The rejection region comprises values of the test statistic leading to the rejection of the null hypothesis.
• A Type I Error means rejecting a true null hypothesis (False Positive), with probability $\alpha$.
• A Type II Error means failing to reject a false null hypothesis (False Negative), with probability $\beta$.
• The power of the test is the probability of rejecting a false null hypothesis, calculated as $1 - \beta$.
• The P-value is the probability of observing a test statistic as extreme as, or more extreme than, the one observed, assuming $H_0$ is true.

Common Hypothesis Tests

• For the mean $\mu$ of a normal population with known variance $\sigma^2$, the test statistic is $Z = \frac{\bar{x} - \mu_0}{\sigma/\sqrt{n}}$.
• For the mean $\mu$ of a normal population with unknown variance $\sigma^2$, the test statistic is $T = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}$.
• For the variance $\sigma^2$ of a normal population, the test statistic is $\chi^2 = \frac{(n-1)s^2}{\sigma^2_0}$.
• For the proportion $p$ of a binomial population, the test statistic is $Z = \frac{\hat{p} - p_0}{\sqrt{p_0(1 - p_0)/n}}$.

Steps in Hypothesis Testing

• State the null ($H_0$) and alternative ($H_1$) hypotheses.
• Choose the significance level $\alpha$.
• Determine the test statistic and its distribution under $H_0$.
• Define the rejection region.
• Calculate the test statistic from the sample data.
• Decision: Reject $H_0$ if the test statistic is in the rejection region or if the p-value $ < \alpha$; otherwise, fail to reject $H_0$.

Example: T-Test

• Problem: Test if the mean student height is 170 cm, given a sample of 25 students with $\bar{x} = 172$ cm and $s = 5$ cm, using $\alpha = 0.05$.
• $H_0: \mu = 170$, $H_1: \mu \neq 170$.
• $\alpha = 0.05$.
• Test statistic: $T = \frac{\bar{x} - \mu_0}{s/\sqrt{n}} = \frac{172 - 170}{5/\sqrt{25}} = 2$.
• Rejection region: $|T| > t_{\alpha/2, n-1} = t_{0.025, 24} = 2.064$.
• Decision: Since $|2| < 2.064$, fail to reject $H_0$.
• Conclusion: There is insufficient evidence to say mean student height differs from 170 cm.

Explanation of point estimation in statistical inference, covering estimators, unbiasedness, efficiency and consistency. Methods for finding estimators, including the method of moments and maximum likelihood estimation (MLE) are explored. Includes likelihood and log-likelihood functions.