K-Nearest Neighbors (KNN) Algorithm

K-Nearest Neighbors (KNN) Classifier

• KNN is a simple algorithm that stores all available cases and classifies new cases based on a similarity measure (distance function).
• A case is classified by a majority voting of its neighbors, with the case being assigned to the class most common among its K nearest neighbors measured by a distance function.

Euclidean Distance

• Euclidean distance is a common distance function used in KNN classification.
• The Euclidean distance between two points (x1, y1) and (x2, y2) is calculated as √((x2 - x1)^2 + (y2 - y1)^2).

KNN Example

• Points are classified as GOOD or BAD based on their features (Durability and Strength).
• P5 (3,7) is classified as GOOD based on the majority voting of its 3 nearest neighbors.
• The 3 nearest neighbors of P5 are P3 (3,4), P4 (1,4), and P1 (7,7).

KNN Pseudocode

• No specific pseudocode is provided, but the general algorithm is described.

Unsupervised Methods

• In unsupervised classification, the class labels are unknown, and the data are plotted to see whether they cluster naturally.
• If K=1, then the case is simply assigned to the class of its nearest neighbor.

k-Nearest-Neighbor (k-NN) Classifier

• k-NN classifier uses a distance function to find the k nearest neighbors of a new input vector x.
• The object is assigned to the most frequently occurring class among its k nearest neighbors.

Parametric and Non-Parametric Methods

• Parametric methods involve maximizing the posterior probability, P(ωi|x), which is the same as maximizing p(x|ω1).P(ωi)).
• Non-parametric methods do not require making assumptions about the population parameters.
• Examples of non-parametric methods include KNN, Decision Tree Model, etc.