SVM Algorithm PDF

08/01/2025 Support Vector Machine (SVM) Algorithm Course: Machine learning Chapter: SVM SVM is a powerful machine learn...

08/01/2025 Support Vector Machine (SVM) Algorithm Course: Machine learning Chapter: SVM SVM is a powerful machine learning algorithm widely used for both linear and nonlinear classification, as well as regression and detection tasks. Engineering program Cybersecurity – Semester 3 SVMs are highly adaptable, making them suitable for various applications such as text classification, image classification, spam detection, handwriting identification, face detection, and anomaly detection. SVMs are particularly effective because they focus on finding the maximum separating hyperplane S. LAZAAR between the different classes in the target feature, ENSA of Tangier 2024-2025 making them robust for both binary and multiclass classification. 1 2 Hyperplane A Scenario: Definition In the figure we have one blue ball in the boundary of Mathematically, a hyperplane can be defined as the set the red ball. of points x in an n-dimensional space satisfying the linear equation: How does SVM classify the data? w⋅x+b=0 The blue ball in the boundary of red ones is an outlier of blue balls. w is a vector normal (perpendicular) to the hyperplane. The SVM algorithm has the characteristics to ignore b is the bias term. the outlier and finds the best hyperplane that maximizes the margin. x represents points in the space. Example: A hyperplane is a geometric concept commonly used in mathematics and machine learning. * An outlier is a data point that significantly differs from the 1. In 2D space (a plane), a hyperplane is a line. majority of observations in a dataset. 2. In 3D space (a volume), a hyperplane is a plane 3 4  08/01/2025 How does Support Vector Machine Algorithm Work? Examples A good choice for the best hyperplane in a Support Vector Machine is the one that maximizes the separation margin between the two classes. The maximum-margin hyperplane, also referred to as the hard margin, is selected based on maximizing the distance between the hyperplane and the nearest data point on each side. We choose the hyperplane whose distance from it to the nearest data point on each side is maximized. From the figure, we choose L2. 5 6 How does Support Vector Machine Algorithm Work? INITIAL DISTRIBUTION Problem How to choose the hyperplane whose distance from it to the nearest data point on each side is maximized, OPTIMAL SOLUTION Hyperpla ne POSSIBLE SOLUTIONS 7 8  08/01/2025 Mathematical Computation Mathematical Computation Soft-Margin SVM: Hard-Margin SVM (Linearly Separable Data): * The constraint ensures that all points are correctly classified. 9 10 Types of Support Vector Machine Based on the nature of the decision boundary, SVM can be divided into two main parts: * In the context of SVMs, a slack variable is introduced to handle cases where data Linear SVM: use a linear decision boundary to separate the data points of different classes. points violate the margin or even fall on the wrong side of the decision boundary. When the data can be precisely linearly separated, linear SVMs are very suitable. This means that a single straight line (in 2D) or a hyperplane (in higher dimensions) can entirely divide the If the point is correctly classified and within the margin, the slack variable is zero. data points into their respective classes. If the point is misclassified or outside the margin, the slack variable takes a positive value. Non-Linear SVM: can be used to classify data when it cannot be separated into two classes by a straight line (in the case of 2D): By using kernel functions, nonlinear SVMs can handle nonlinearly separable data. The original input data is transformed by these kernel functions into a higher-dimensional feature space, where the data points can be linearly separated. * Slack: marge de tolérance ou écart. A linear SVM is used to locate a nonlinear decision boundary in this modified space. 11 12  08/01/2025 Illustration Nonlinearly separable data Linearly separated KERNEL FUNCTIONS https://www.youtube.com/watch?v=3liCbRZPrZA 13 14 Kernel function is a Types of kernel functions: mathematical function used This transformation allows to transform data into a SVM to find a hyperplane higher-dimensional that can classify data that is space where it becomes not linearly separable in the easier to separate using a original input space. linear decision boundary. 15 16  08/01/2025 Types of kernel functions: Types of kernel functions: 17 18 Types of kernel functions: 19