Measurement Scales and Hypothesis Testing

Measurement Scales of Data

• Quantitative data consists of numerical values. It can be:
  • Discrete: Whole numbers that cannot be divided, like the number of babies.
  • Continuous: Numbers that can be divided, like height.
  • Interval: Data possessing properties of nominal and ordinal data, where differences between values are calculable, exemplified by Celsius or Fahrenheit temperatures.
  • Ratio: Possesses all properties of interval data, including a true zero value, allowing ratios to be determined, like height.
• Qualitative data describes data qualities using non-numerical values.
  • Nominal: Data is categorized without numerical meaning nor specific order, such as employed or unemployed status.
  • Ordinal: Data is placed in a specific order, such as low, medium, or high socioeconomic status.

Hypothesis Testing

• Sampling variability refers to the variation in calculated means and standard deviations among different samples.
• Null hypothesis significance testing is a method of statistical inference, in which an experimental factor is tested against a hypothesis of no effect or relationship.
• A p-value signifies the probability of obtaining a result at least as extreme as what was observed, assuming the null hypothesis is accurate.
• Confidence intervals (CI) provide a range of values indicating the level of confidence in an estimate, such as a 95% CI.

Simple Linear Regression

• A statistical model is a mathematical representation of the relationship between variables.
• Linear association refers to the relationship between two numeric variables which can be represented using a straight line.
• Correlation measures the direction and strength of a relationship between two numeric variables, ranging between -1 and 1.
• Simple linear regression relates X to Y through an equation: Y = β0 + β1X + ε.
• OLS (ordinary least squares) criterion finds the best-fit line.
  • β0: Intercept.
  • β1: Slope.
  • ε: Error term.
• Null hypothesis significance testing (NHST) tests the significance of regression parameters.
• Goodness of fit (GOF) is interpreted using R-squared.
• Partitioning of variation (Sum of Squares - SS) divides total variation into explained and unexplained variation.

Centering of Continuous Predictors

• Centering involves subtracting the mean of experience from each subject's experience, resulting in a new mean of 0.
• The intercept of the regression model then represents the expected salary for a subject with experience equal to the sample mean.

Assumptions for Linear Regression

• The residuals should be normally distributed.
• Homoscedasticity, which assumes a constant variance of errors.
• Independence of error terms.
• Linearity.

Standardized Residual Plots

• A good plot has no patterns, with errors evenly distributed around 0, no clear outliers, and consistent variance.
• Normality is checked by plotting quantiles of residuals against theoretical quantiles, with points ideally following a diagonal line.

Goodness of Fit

• R-squared measures how well the model fits the data and represents the proportion of variation in the dependent variable explained by the independent variable.
• R-squared ranges from 0 to 1, with higher values indicating a better fit.

Objective and Idea of Linear Regression

• Linear regression aims to predict salary (Y) based on years of experience (X).
• The goal is to fit a line through the data points that minimizes the sum of squared differences between the data points and the regression line (residuals).
• Simple Linear Regression Model: 𝒀𝒊 = 𝜷𝟎 + 𝜷𝟏𝑿𝒊 + 𝝐𝒊
  • Y represents salary.
  • X represents years of experience.
  • 𝛽0 is the intercept, representing average salary when X = 0.
  • 𝛽1 is the slope, representing change in average salary with a unit increase in X.
  • 𝝐 is the residual, representing deviation from the expected salary value.
• Formulating Hypotheses:
  • Hypothesis/Research Question: Can years of experience influence salary?
  • Regression Equation: 𝒀𝒊 = 𝜷𝟎 + 𝜷𝟏𝑿𝒊 + 𝝐𝒊
  • Null Hypothesis: 𝐻0: 𝛽1 = 0
  • Alternative Hypothesis: 𝐻1: 𝛽1 ≠ 0
• Hypothesis Testing can be conducted in three ways:
  • Check if the p-value is smaller than 0.05
  • Check if the confidence interval includes zero
  • Check if the t-value is among the 5% of most unlikely values of the central t-distribution with 𝑑𝑓 = 448 degrees of freedom
  • If one statement is true, all statements are equivalent

Multiple Linear Regression

• Problem: You cannot use "words" or categories as variables in the regression model
• Solution: Variables should be recoded into numeric values using dummy variables

Linear Regression for Binary Outcomes

• Problem: Linear regression is not suitable for binary outcomes because the dependent variable ranges between 0 and 1, while linear regression can predict values between -∞ and ∞ and residuals are not normally distributed
• Logistic Regression: Models the logit (log-odds) of the probability of an event occurring.

Logistic Regression

• Logistic regression is used for binary outcome variables, modeling the probability that a given input point belongs to a particular category.
• The odds ratio measures the association between an exposure and an outcome. It represents the odds that an outcome will occur given a particular exposure, compared to the odds of the outcome occurring without that exposure.
• A binary variable has two possible outcomes, often coded as 0 and 1.
• A categorical variable can take on one of a limited number of values, assigning each individual to a particular group or nominal category.
• A continuous variable can take on an infinite number of values within a given range.
• Residuals represent the difference between observed and predicted values.
• Standard error measures the statistical accuracy of an estimate, indicating the fluctuation of the sample mean due to random sampling.
• A p-value helps determine the significance of hypothesis testing results, where a p-value less than 0.05 is often statistically significant.
• A confidence interval (usually 95%) is a range of values likely to contain the population parameter.
• Maximum Likelihood Estimation (MLE) estimates the parameters of a statistical model by maximizing the likelihood of the observations.
• Goodness of fit measures how well a statistical model fits a set of observations; common tests include the Chi-square test and the Hosmer-Lemeshow test.

Descriptive Statistics

• Descriptive Statistics are used to summarize and describe the main features of a Data set
  • Mean: The average of a set of numbers
  • Median: The middle value in a set of numbers
  • Mode: The most frequently one is a set of numbers
  • Standard Deviation (SD): A measure of the amount of variation or dispersion in a set of values
  • Proportion: Type of ratio that represents a part of the whole, often used for categorical data

Steps in Calculating Descriptive Statistics

• Mean (average): Sum all numbers in the dataset, then divide by the total number of values (n)
• Median: Arrange the numbers in ascending order; if the number of values (n) is odd, the median is the middle number; if even, the median is the average of the two middle numbers -Mode: Identify the number that appears most frequently in the dataset; a dataset can have multiple modes
• Standard Deviation: (\text{SD} = \sqrt{\frac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n-1}}) calculate the mean (\bar{x} ) of the dataset; subtract the mean from each number to find the deviation; square each deviation; sum all squared deviations; divide the sum by the number of values minus one (n-1) to get the variance; take the square root of this variance to get the standard deviation

Inferential Statistics involve:

• Using Sample Population for generalization by using:
  • Confidence Interval (CI): A range of values that is likely to contain the population parameter with a specified confidence level (e.g., 95% CI)
• Probability of obtaining test results at least as extreme as the observed ones, assuming the null hypothesis is correct - Odds Ratio (OR): A measure of Exposure association in relation to the outcome, higher then 1, the event in question will me more likely with the first group
• A statistical method used to model the relationship between a binary dependent variable and one or more independent variables to estimate the odds of a certain event occurring
• Confounding: A situation in which the effect of the primary independent variable on the dependent variable is mixed with the effect of another variable. Interaction Effects: Occur when the effect of one independent variable on the dependent variable changes

Lecture Questions - Lecture 1

• Statistical models aim to simplify reality, capture key relationships, and provide insights.
• Simple Linear Regression Equations: Yi=β0+β1Xi+ϵi

Lecture Questions - Lecture1 - Terms

• β0 represents the intercept of value when X=0
• β1\beta_1β1 is the slope (the change in Y for a unit change in X)
• ϵi\epsilon_iϵi is the error term (captures randomness or deviations from the fitted line)
• Β0: Represents Dependent Valuable when independent variable is zero
• β1: Represents a change in one Increase in in the independent valuable

Regression Analysis

• Null hypothesis: There is no linear relationship B1=0
• R-squared: The proportion of variance in the dependent variable explained by the independent variable.
• OLS Assumption: Residuals are homoscedastic meaning consistent variance across all fitting values
• Centering: To subtract the mean from every value of the independent value

Lecture Questions - Residual Plot

• Good residual plots are randomly scattered around zero without discernible trends.
• Low P Values reject the Null Hypothesis
• Centering Changes The interpretation of the intercept to represent the mean value
• High R- squared imply a high level of variance in Y is explained by X
• Normality is important to check in order to ensure valid inference from hypothesis test
• Homoscedasticity is violated if residuals increase in magnitude as fitted values increase

What does the intercept in a linear regression model represent

• The value of the outcome value when all predictors are zero
• Confounding: Confounding variables distort the perceived effect of another variable due to its association with both the dependent and independent variable

How interaction and Confounding effects differ

• Interaction: Effect indicated that the effect of one variable depends on the level of another
• While a confounding variable affects distorts the true relationship of variables

Multiple regression

• In multiple regression adding interaction is to allow the model to account relationships where the effect that one explanatory variable depends on another
• Benefits for reducing a categorical variable into dummy variables in a regression model is that it allows the conclusion of categorical data is a a regression analysis by converting categories into numerical data

Why are CI preferred over P- Value

• CI’s provide arrange a possible values for the parameter estimate offering more inforfation than a Pvalue
• Multicollinearity: The situation where independent variables are highly correlated with one another
• VIF>5: Indicates high multicollinearity which requires corrective measures

Logistic Regression

• To predict the probability of binary outcome
• A ratio of 0.5 indicates that the event is half as likely in the reference group
• An odds ratio or 1 is in where both groups are the same
• Odd can determine probability of binary outcomes and may probabilities onto the linear infinite scale

What does an odds ratio> 1 signify

• the event is likely to occur in group represented by the numerator
• what is the ratio of O. 67 mean for particular variable
• Probability is less than zero point five

What is the log odds of the reference category

• The Log odds of the reference category when all predicted are zero
• logistic regression models properly limit probability between zero and one

Interaction Terms:

• Including multiple predictor variable
• The overall test is to compare the levels of categorical variable after conduiting an overall test
• Effect for interaction
• What does it mean if there’s a confirming factor is what
  • Confirming can mass cord exaggerate the true relationship between independent variables
  • Stratification to provide clearer interpretation in mean the presence of interaction