ISLR Chapter 4 Flashcards

Qualitative Variables

• Defined as variables without numeric value; examples include eye color and gender.

Classification

• Refers to predicting qualitative responses based on input data.

Classifier

• A generic term encompassing various classification techniques used to categorize data.

Logistic Regression

• A classification technique modeling the probability of a response belonging to a specific class using a sigmoid function.

Linear Discriminant Analysis (LDA)

• A classifier that assumes predictors follow a Gaussian distribution; it uses Bayes' Theorem to determine the most probable class for a given observation.

Binary Response

• A type of response variable with only two possible outcomes.

k-Nearest Neighbors (k-NN)

• A non-parametric classification method that classifies observations based on majority class membership of nearby data points.

Logistic Function

• Mathematically expressed as $\frac{e^{\beta X}}{1 + e^{\beta X}}$, representing the sigmoid curve used in logistic regression.

Maximum Likelihood

• A parameter estimation method where the likelihood function is maximized to fit a statistical model.

Odds

• Defined as $\frac{P(X)}{1 - P(X)}$, a measure of the likelihood of an event occurring relative to it not occurring.

Log-Odds / Logit

• Expressed as $\ln{\frac{P(X)}{1 - P(X)}}$, representing the logarithm of the odds.

Confounding

• Occurs when correlations among predictors skew the perceived relationships between them and the response variable.

Prior Probability

• Refers to the likelihood that a randomly selected observation belongs to a specific class before considering any evidence.

Density Function

• A continuous function, denoted as f(x), describing the probability of a variable x occurring.

Bayes' Theorem

• A foundational concept in probability, expressed as $p(A and B) = P(B | A)P(A)$, relating conditional probabilities.

Posterior Probability

• Represents the probability of a class given observed evidence, denoted as $f(a) = p(A = a | B)$.

Normal Gaussian Distribution

• Mathematically defined as $\frac{1}{\sqrt{2\pi}\sigma} \exp{( -\frac{1}{2\sigma^{2}} (x-\mu_{k})^{2} )}$, characterizing the bell-shaped curve of typical distribution.

Discriminant Function

• Utilizes auxiliary functions to estimate the class most likely associated with a test observation.

Multivariate Gaussian

• Extends Gaussian distribution to multiple dimensions, encompassing p > 1 variables.

Overfitting

• Occurs when a model captures noise from the training data rather than the underlying relationship, potentially leading to poor performance on new data.

Null Classifier

• A baseline classifier assigning observations to the most frequent class in the training set without additional information.

Confusion Matrix

• A comprehensive representation of classifier performance, summarizing true positive/negative and false positive/negative counts.

Sensitivity

• A crucial classifier metric measuring the proportion of true positives identified correctly.

Specificity

• A metric that represents the proportion of true negatives accurately identified by the classifier.

Receiver Operating Characteristic (ROC) Curve

• A graphical representation plotting true positive rate against false positive rate across various threshold settings to evaluate classifier performance.

Quadratic Discriminant Analysis

• A model that assumes observations from each class originate from Gaussian distributions with distinct covariance matrices for each class.

Test your knowledge of key terms from Chapter 4 of ISLR. This set of flashcards covers essential concepts such as qualitative variables, classification techniques, and logistic regression. Perfect for anyone studying data science and statistics.