Principal Component Analysis (PCA) Overview

A statistical technique used to find patterns and correlations between multiple variables within a dataset. It aims to simplify complex datasets by discovering the major trends and relationships.

A test that checks if the variables in a dataset are uncorrelated. It assumes the population correlation matrix is an identity matrix, meaning variables are independent.

PCA is particularly useful when dealing with datasets containing a large number of variables. It reduces the complexity of the data by extracting the most important information.

A measure of how well the variables in a dataset are related. A higher correlation coefficient indicates a stronger relationship between variables.

PCA works by creating linear combinations of the original variables. These combinations, called principal components, maximize the variance of the data, capturing most of the information.

The first principal component (PC1) accounts for the most variance, followed by PC2, PC3, and so on, with each subsequent component capturing less variance.

The number of variables in a dataset influences the effectiveness of PCA. More variables mean more factors to potentially reduce.

PCA helps to reduce the dimensionality of data by transforming correlated variables into uncorrelated ones. This means the new variables are independent and don't share the same information, reducing redundancy.

PCA is a powerful technique because it can uncover hidden relationships within data. By examining the principal components, you can identify the most important factors driving the patterns in your data.

The ratio of sample size to the number of variables in a dataset. This ratio impacts the reliability of PCA results.

PCA is not scale invariant. Standardizing variables ensures that they have the same unit of measurement before analysis.

PCA finds the major correlations within the data by identifying the most significant linear combinations of the original variables. These relationships are used to create the principal components.

PCA is known for its ability to reduce data loss during the transformation process. This makes it a valuable tool for maintaining as much information as possible while simplifying the data.

PCA is widely used in different fields, including finance, bioinformatics, and machine learning, because of its ability to analyze complex data and extract key insights.

A matrix representing the relationships between variables, showing how much they vary together.

A numerical value that indicates how much a variable changes in relation to another variable.

A special kind of matrix where all numbers on the diagonal are the same, and the numbers above and below the diagonal are mirrored.

A type of matrix where all eigenvalues are non-negative (zero or positive).

The result of a mathematical operation that breaks down a complex dataset into simpler, independent components.

A mathematical process that finds the directions of greatest variance in data.

A number associated with an eigenvector, indicating the amount of variance explained by that eigenvector.

Measures how much the variance of the data is captured by the principal component. It is the maximum eigenvalue of the covariance matrix.

The first principal component (PC1) is the direction in the data with the highest variance. It's the eigenvector that corresponds to the largest eigenvalue of the covariance matrix.

The remaining principal components (PC2, PC3, etc.) are generated in the same way as PC1 but with decreasing variance. Each PC is an eigenvector of the covariance matrix, and their corresponding eigenvalues are in descending order.

The goal of PCA is to reduce the dimensionality of the data. It seeks to find a smaller set of principal components that capture most of the variability in the original data.

Each principal component is a direction in the data. The number of principal components is equal to the number of original variables or less.

PCA assumes that the data is linearly related. It is a linear transformation of the original variables, meaning that the relationships between the variables are linear.

The total variance in the data is conserved after PCA. The sum of the variances of all original variables equals the sum of the variances of all principal components.

Principal components are orthogonal to each other, meaning they are independent. This ensures that each PC captures a unique aspect of the data.

Principal Component Analysis (PCA) is a technique that aims to reduce the dimensionality of complex data by finding a smaller set of variables, known as Principal Components, that capture most of the variation in the original data. In essence, it seeks to find the most important directions in your data.

There can be at most 'p' principal components where 'p' is the number of original variables.

Each Principal Component (PC) is a linear combination of the original variables, defined by coefficients called Loadings. This means that each PC is a weighted sum of the original variables.

The variance of a principal component is calculated by summing the squared weighted covariances between the original variables. This essentially tells you how much diversity or spread the PC captures from the original data.

The first principal component (PC1) is chosen to maximize variance, capturing the most variability in the original variables. This means that PC1 is the direction in the data space where the data points spread out the most.

The goal of PCA is to find the linear combination of original variables (PC) that maximizes variance, meaning finding the direction in the data space where the data points are most scattered.

Component loadings ( ) are the correlations between the variables (rows) and components (columns).

The squared component loading is the percent of variance in that variable that is explained by the component.

A loading higher than 0.3 indicates that the variable accounts for 0.09 (i.e., 0.3 squared) of the variance in the component.

Communality represents the amount of variance of the original variable that is summarized by the retained components.

1 - h is the amount of variance discarded by selecting only a specific number of components.

For each variable, communality is the sum of the squared loadings across the retained components.

The steps in PCA include: 1. Checking for PCA adequacy, 2. Extracting principal components and determining their number, 3. Rotating the components and interpreting them, 4. Making a final decision on the number of underlying components, and 5. Creating scores.

PCA adequacy is assessed before proceeding with the analysis.