Module 3:Data Preprocessing: Reduction and Transformation

• The lecture discusses various methodologies and approaches to prepare and clean data for analysis.
• The topics include data reduction, data transformation, data discretization, and measuring data similarity and dissimilarity.
• The lecture also covers preprocessing practices on Weka.

Data Reduction Strategies

• Data reduction aims to obtain a smaller, reduced data representation that produces similar analytical results.
• Databases and data warehouses may store terabytes of data, making complex data analysis time-consuming.
• Data reduction strategies include:
  • Dimensionality reduction (removing unimportant attributes)
  • Wavelet transforms
  • Principal Components Analysis (PCA)
  • Feature subset selection
  • Feature creation
  • Numerosity reduction (regression, log-linear models, histograms, clustering, sampling, etc.)
  • Data cube aggregation
  • Data compression

Dimensionality Reduction

• The "curse of dimensionality" refers to increased data sparsity as dimensionality increases.
• Density and distance between points, which are critical to clustering and outlier analysis, become less meaningful.
• The possible combinations of subspaces grow exponentially.
• Dimensionality reduction avoids the curse of dimensionality, helps eliminate irrelevant features and noise, reduces time and space requirements, and allows easier visualization.
• Dimensionality reduction techniques include wavelet transforms, Principal Component Analysis, and supervised/nonlinear techniques (e.g., feature selection).

Wavelet Transform

• A wavelet transform decomposes a signal into different frequency sub-bands.
• It's applicable to n-dimensional signals.
• Wavelet transforms preserve the relative distance between objects at different levels of resolution.
• Wavelet transforms allow natural clusters to become more distinguishable and are used for image compression.

Wavelet Transformation Details

• Discrete Wavelet Transform (DWT) is used for linear signal processing and multi-resolution analysis.
• DWT provides compressed approximations by storing a small fraction of the strongest wavelet coefficients.
• DWT is similar to Discrete Fourier Transform (DFT) but provides better lossy compression and is localized in space.
• The length of input data must be an integer power of 2; otherwise, it is padded with 0s.
• Has two smoothing functions
• Applies transforms iteratively

Wavelet Decomposition Example

• A math tool for decomposing functions
• An example data set S = [2, 2, 0, 2, 3, 5, 4, 4] can be transformed.
• Small detail coefficients in wavelet decomposition can be replaced by 0s for compression while retaining significant coefficients.

Haar Wavelet Transform

• Haar wavelet transform is a type of wavelet transform.
• It can be represented by a hierarchical decomposition structure ("error tree").

Wavelet Transform Advantages

• Using hat-shape filters:
• Region Emphasized
• Suppress weaker information
• Effective removal of outliers.
• Reduced sensitivity
• Multi-resolution
  • Detect arbitrary shaped clusters
• Efficient with Complexity O(N).
• Particularly useful when the dimensionality of the data is low

PCA

• Principal Component Analysis finds a projection that captures the largest amount of variation in data.
• PCA projects the original data into a much smaller space, thereby reducing dimensionality.
• Eigenvectors of the covariance matrix define this new space.

Attribute Subset Selection

• Serves as another method to reduce dimensionality of data.
• Redundant attributes duplicate information contained in other attributes.
  • This includes duplicate data
• Irrelevant attributes provide no useful information for the data mining task at hand.
  • Irrelevant information like ID numbers when predicting GPA

Heuristic Attribute Selection

• There are 2^d possible attribute combinations for d attributes.
• Typical heuristic attribute selection methods include:
• Single attribute selection based on attribute independence.
• Step-wise feature selection
  • Finding the best single attribute
• Attribute elimination
  • Finding the worst attribute
• Combined attribute selection
• Branch and bound by eliminating and backtracking.

Attribute Creation (Feature Generation)

• Create new, more effective is the main goal.
• Methodologies:
  • Attribute extraction (domain-specific)
  • Mapping data to new space (Fourier/wavelet transformation)
  • Attribute construction involves discriminatory work

Numerosity Reduction

• Reduces data volume by choosing alternative, smaller forms.
• Parametric methods (e.g., regression)
  • Assume data fits a model.
  • Store/predict outliers
• Non-parametric methods
  • Histograms, clustering, sampling

Regression and Log-Linear Models

• Linear regression models data to fit a straight line, using the least-squares method.
• Multiple regression models a response variable Y as a linear function of a multidimensional feature vector.
• Log-linear models approximate discrete multidimensional probability distributions.

Regression Analysis Details

• It is a collective name for the modeling and analysis of numerical data, including a dependent variable (response variable) and one or more independent variables (explanatory variables).
• Parameters are estimated to give a "best fit."
• Commonly, the best fit is evaluated using the least squares method.
• It's used for prediction (including forecasting time-series data), inference, hypothesis testing, and causal relationship modeling.

Regression Analysis and Log-Linear Models

• In linear regression, Y = wX + b for the estimated line.
  • It needs two regression
• Multiple regression uses Y = b0 + b1X1 + b2X2, where nonlinear examples become linear.
• Log-linear approximate discrete data.

Histogram Analysis

• Divides data into buckets and stores average or aggregate sums for each bucket.
• Partitioning rules include:
• Equal-width range
• Equal-frequency depth

Clustering

• Partition data sets into clusters based on similarity.
• Store cluster representation (e.g., centroid and diameter).
• Very effective
• It's possible to store hierarchical
• Has many methods
• Analysis of cluster is later

Sampling Data

• Sampling takes a small set to represent a whole set.
• Allows a mining algorithm to work faster than the number of total data.
• Best if the chosen set accurately represents the whole set.
• Simple data will do poorly
• Using adaptive works best.

Types of Sampling

• Simple random sampling:
  • Each items has equal probability of being selected.
• Sampling without replacement has no duplicates.
• Sampling with replacement can have duplicates.
• Stratified sampling partitions data.
  • Draws samples from each to manage skewed data

Data Transformation methods

• A function that maps values to a smaller set of values.
• Data Transformation smoothing: Remove noise from data.
• Attribute/feature construction
• New data constructed Aggregation: Summarization
• Scale to fall within a smaller, specified range
• Concept hierarchy climbing

Normalization

• Min-max
  • Original to new
• Used to normalize data
• Z-score.
  • Useful when the actual minimum and maximum are unknown.

Standardizing Data

• Applying a Z-score is useful when the mean is known.
• This involves calculating from mean/standard deviation
• An alternative way: Calculate/standardize through mean/standard deviation

Discretization Types

• Three attribute types:
• The three types are Nominal, Ordinal, and Numeric.
• Three processes
• Using Labels to separate
• Reduce Data Size
• Split data
• Prepare
  • prepare for classfication

Discretization Methods

• There are some data discretization methods.
• These methods may include split, tree, and corelation

Simple Discretization

• Equal-width (distance) partitioning divides the range into N uniform intervals.
  • If the data is from A is the lowest and B is the highest
  • W = (B-A)/N
• Equal-depth (frequency) partitioning divides the range into N intervals
  • The number are approxamitely the same
  • Categorical can be tricky

Discretization Details

• It is possible to sort data. Applying a Partition is useful when the range is same
• Apply means
• Apply Boundries

Discretization Without Classes

• Binning and clustering
• Clustering leads to better results

Discretization

• Classification is done recursivley
• Bottom up vs Top Down can vary
• Using low values is an example of a Chi-square
• Classification is used

Concept Hierarchy Generation

• Specified through total ordering, schema, or expert data
• Examples of Attributes
• Auto Gen

Automatic Hierarchy Generation

• The number of attributes are placed at the lower level of the hierchy
• Exceptions can occur in the year

Data Similarity

• Similarity is a numerical to see how two data points are alike.
  • Similarity measures how alike
• Dissimilarity measures the dissimilarity between date
  • Distance shows how unrelated two items are
• Proximity refers to a similarity or dissimilarity

Data and Dissimilarity Matrix

• A matrix with p dimensions and two modes.
• A matrix that shows the distance, traingular, and single mode

Proximity Measure for Nominal Attributes

• States with numbers and names
• m total

Binary Attributes

Data Dissimilarity

• Using examples
• Symmetry is equal 1s
• Let Y and P = 1
• N = 0

Data Standardization

• Using Z shows number

Distance in Numeric Data

• distance defined by dimension

Cases for Distance

• Manhattan is block L1
• Hamming bits between the data that is binary
• 2 is L norm

Ordinal Variables

• Can be continuous
• Replaced to intervals of scales
• Use if statements

Mixed Types

• Using combinations of weight formula
• Use binary
• Normalize
• Determine rank

Cosine Similarity

• Documents can have attributes
• Term frequency occurs more then once
• Use feature arrays
• Vectors indicate relationships