ISLR Chapter 4 Flashcards

Questions and Answers

What does qualitative refer to?

Variables without numeric value (e.g. eye color)

What is classification in data analysis?

Predicting qualitative responses

What is a classifier?

Generic name for a classification technique

What is logistic regression used for?

Models the probability of response being in a particular class with a sigmoid function Signup and view all the answers

What does Linear Discriminant Analysis (LDA) assume about predictors?

They are assumed Gaussian Signup and view all the answers

What is a binary response?

Response with two possible levels Signup and view all the answers

What is k-Nearest Neighbors?

Non-parametric classifier which partitions the feature space using majority class membership in local areas Signup and view all the answers

What is the logistic function represented by?

$\frac{e^{\beta X}}{1 + e^{\beta X}}$ Signup and view all the answers

What is maximum likelihood fitting?

A method of fitting parameters to a model by maximizing the likelihood function Signup and view all the answers

What does odds represent?

$\frac{P(X)}{1 - P(X)}$ Signup and view all the answers

What are log-odds / logits?

$\ln{\frac{P(X)}{1 - P(X)}}$ Signup and view all the answers

What does confounding refer to?

When correlations among predictors distort perceived relationships Signup and view all the answers

What is the prior in probability?

The probability that a randomly chosen observation belongs to a certain class Signup and view all the answers

What does a density function do?

Describes the probability of x occurring Signup and view all the answers

What does Bayes' Theorem express?

$p(A and B) = P(B | A)P(A)$ Signup and view all the answers

What is the posterior probability?

$f(a) = p(A = a | B)$ Signup and view all the answers

What is the normal Gaussian distribution formula?

$\frac{1}{\sqrt{2\pi}\sigma} \exp{( -\frac{1}{2\sigma^{2}} (x-\mu_{k})^{2} )}$ Signup and view all the answers

What is a discriminant function?

Auxiliary functions used for determining the class with highest probability of membership Signup and view all the answers

What defines a multivariate Gaussian?

Gaussian with p > 1 Signup and view all the answers

What is overfitting in modeling?

When a model learns the noise in the training data Signup and view all the answers

What is a null classifier?

Classifies based on only the prior Signup and view all the answers

What is a confusion matrix?

Generalization of the True/False Positive/Negative Table Signup and view all the answers

What does sensitivity measure in classification?

Proportion of true positives identified Signup and view all the answers

What does specificity measure in classification?

Proportion of true negatives identified Signup and view all the answers

What is the Receiver Operating Characteristic (ROC) curve?

Plot of (true positive, false positive) pairs for different cutoff thresholds Signup and view all the answers

What is quadratic discriminant analysis?

Model that assumes observations from each class are drawn from a Gaussian distribution with their own covariance Signup and view all the answers

Study Notes

Qualitative Variables

Defined as variables without numeric value; examples include eye color and gender.

Classification

Refers to predicting qualitative responses based on input data.

Classifier

A generic term encompassing various classification techniques used to categorize data.

Logistic Regression

A classification technique modeling the probability of a response belonging to a specific class using a sigmoid function.

Linear Discriminant Analysis (LDA)

A classifier that assumes predictors follow a Gaussian distribution; it uses Bayes' Theorem to determine the most probable class for a given observation.

Binary Response

A type of response variable with only two possible outcomes.

k-Nearest Neighbors (k-NN)

A non-parametric classification method that classifies observations based on majority class membership of nearby data points.

Logistic Function

Mathematically expressed as $\frac{e^{\beta X}}{1 + e^{\beta X}}$, representing the sigmoid curve used in logistic regression.

Maximum Likelihood

A parameter estimation method where the likelihood function is maximized to fit a statistical model.

Odds

Defined as $\frac{P(X)}{1 - P(X)}$, a measure of the likelihood of an event occurring relative to it not occurring.

Log-Odds / Logit

Expressed as $\ln{\frac{P(X)}{1 - P(X)}}$, representing the logarithm of the odds.

Confounding

Occurs when correlations among predictors skew the perceived relationships between them and the response variable.

Prior Probability

Refers to the likelihood that a randomly selected observation belongs to a specific class before considering any evidence.

Density Function

A continuous function, denoted as f(x), describing the probability of a variable x occurring.

Bayes' Theorem

A foundational concept in probability, expressed as $p(A and B) = P(B | A)P(A)$, relating conditional probabilities.

Posterior Probability

Represents the probability of a class given observed evidence, denoted as $f(a) = p(A = a | B)$.

Normal Gaussian Distribution

Mathematically defined as $\frac{1}{\sqrt{2\pi}\sigma} \exp{( -\frac{1}{2\sigma^{2}} (x-\mu_{k})^{2} )}$, characterizing the bell-shaped curve of typical distribution.

Discriminant Function

Utilizes auxiliary functions to estimate the class most likely associated with a test observation.

Multivariate Gaussian

Extends Gaussian distribution to multiple dimensions, encompassing p > 1 variables.

Overfitting

Occurs when a model captures noise from the training data rather than the underlying relationship, potentially leading to poor performance on new data.

Null Classifier

A baseline classifier assigning observations to the most frequent class in the training set without additional information.

Confusion Matrix

A comprehensive representation of classifier performance, summarizing true positive/negative and false positive/negative counts.

Sensitivity

A crucial classifier metric measuring the proportion of true positives identified correctly.

Specificity

A metric that represents the proportion of true negatives accurately identified by the classifier.

Receiver Operating Characteristic (ROC) Curve

A graphical representation plotting true positive rate against false positive rate across various threshold settings to evaluate classifier performance.

Quadratic Discriminant Analysis

A model that assumes observations from each class originate from Gaussian distributions with distinct covariance matrices for each class.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

Test your knowledge of key terms from Chapter 4 of ISLR. This set of flashcards covers essential concepts such as qualitative variables, classification techniques, and logistic regression. Perfect for anyone studying data science and statistics.