Introduction to Pattern Recognition

Introduction to Pattern Recognition

• Pattern recognition involves automatically discovering data regularities using computer algorithms.
• The use of these regularities allows for actions like classifying data into different categories.
• Pattern recognition classifies data based on gained knowledge or statistical information extracted from patterns and their representations.

Applications of Pattern Recognition

• Used in speech recognition.
• Used in Image Processing.
• Used in Fingerprint Identification.
• Used in Optical Character Recognition (OCR).
• Used in Computer-aided Diagnosis.
• Used in Data Mining and Knowledge Discovery.
• Used in Industrial Workflows, including quality control and sorting.

The General Pipeline of Pattern Recognition

• Data Acquisition involves the use of transducers like camera, microphone or sensors.
• Pre-processing involves removing noise from the data.
• This includes image transformation, scaling, rotation, normalization, filtration, and enhancement.
• Segmentation isolates objects from one another and the background, for example flowers.
• Feature Extraction involves finding a new representation in terms of best and strongest features.
• A single flower image with width, length, color, etc. is used.
• Data Reduction is used to reduce curse dimensionality.
• Classification involves classifying features with a trained classifier.
• Classifications may be binary, multiclass, or multi-label.
• Evaluation measures the performance error rate, speed, cost, and robustness.

Pre-Processing

• Raw data needs to be processed and converted for machine use in pattern recognition.
• All forms of multimedia data (Audio, Video, Images, Text) may be converted into a vector of feature values.
• A median mask removes shot noise, like salt-and-pepper noise.
• Variable background brightness and histogram equalization ensure even illumination.
• It's important to handle missing data and detect/handle outlier data.

Segmentation

• It partitions an image into meaningful regions.
• Foreground, comprises the objects of interest.
• Background, is everything else.
• Region-based methods detect similarities using thresholding, such as Otsu, isodata, or maximum entropy thresholding.
• Boundary-based methods detect discontinuities by detecting discontinuities and linking edges to continuous forms like canny detector.

Post-Processing

• Segmented images are post-processed to prepare them for feature extraction.
• Partial objects around the image periphery are removed.
• Disconnected objects can be merged.
• Objects smaller or larger than size limits can be removed, or holes can be filled using morphological opening or closing.

Feature Extraction

• Features are characteristic properties of objects.
• Values are similar for objects in a particular class.
• Values are different from objects in another class or the background.
• Features are continuous, with numerical values like length, area, and texture.
• Features are categorical, with labeled values as ordinal, where the order has meaning such as military rank or satisfaction level.
• Features can be nominal, where the label order isn't meaningful, such as name, zip code, or department.

Problem Analysis for Iris Flower Classification

• Setting up a camera is required to take sample images.
• Characteristics need to be extracted to differentiate species.
• These characteristics are sepal length, sepal width, petal length, petal width and color.

Classification with Sepal Features

• Sepal length can be a feature of discrimination.
• Sepal length alone is a poor classification feature.
• Using single threshold does not unambiguously discriminate between two categories.
• Using only length will result is some errors.
• Relationship between Decision boundary and cost is important with respect to the features.
• Moving the decision boundary to smaller width will reduce any costs.
• Reducing number of virginica will reduce misclassification as versicolor.

Sepal Length and Width in Classification

• Adopting length (x1) and adding the width (x2) of the sepal of the flower improves classification.
• The dark line may serve as a decision boundary of the classifier.
• Overall classification error is lower than using only one feature, but errors can still happen.
• Adding other features can result in (noisy features) that are not correlated with width or length features.
• The best decision boundary is the one which provides an optimal performance.
• Correct categorization of new, different examples from the used-for-training set is generalization.
• Models that are very complex lead to decision boundaries that are complicated, resulting in bad results.

Evaluating Classifiers

• Assessing the overall classification performance comes down to Recall and Precision.

Precision

• The is the fraction of positive predictions that are indeed positive.
• Confusion Matrix - helps classify outcomes,
• A positive case is an outcome where the model is correctly predicted the positive class
• A negative case is an outcome where the model is correctly predicted the negative class
• A false positive case is an outcome where the model has incorrectly predicted the positive class.
• A false negative is an outcome where the model has incorrectly predicted the negative class.
• Precision - Formula : # true positives/ # true positives + # false positives
• It is a continuous measurement from 0 to 1, where one is the best, and zero is the worst.

Recall

• This is a fraction of the positive data that the model predicted to be positive.
• Recall - Formula: # true positives/ # true positives + # false negatives
• It is a continuous measurement from 0 to 1, where one is the best, and zero is the worst.

Accuracy

• Accuracy is the fraction of predictions that the model got right.
• Accuracy = # of correct predictions/ Total number of predictions
• The formula in terms of positives and negatives: TP+TN/ TP+TN+FP+FN
• Accuracy alone isn't detailed enough when working with a class-imbalanced data set where the positive labels differ.

Optimistic vs Pessimistic Model

• The optimistic model has high recall but low precision since almost everything is positive.
• The pessimistic model has high precision but low recall since positive predictions are only made when very sure.

Precision, Recall and F1 scoring

• A pessimistic model finds all positive sentences, and results in many false positives.
• An optimistic model finds few positives, and results in fewer for false positives.
• The goal is to find many positives, but minimize incorrect predictions.
• The trade off between precision and recall is the basic classifier.
• Precision-Recall curve represents the trade off between both.
• F1 Score helps summarize the balance with P/R Numbers, if P or R= 0, then F1 will also equal 0.

Limitations of Pattern Recognition

• Humans switch rapidly and seamlessly between pattern recognition tasks, whereas models cannot.
• Creating a device capable of different classifications like a human is difficult.
• No technique or model suits all pattern recognition problems.