## 12 Questions

What is the main purpose of using Support Vector Machines (SVMs) for classification?

To minimize the cost function and find the optimal hyperplane that separates the classes with the maximum margin.

What is the difference between the cost function used in logistic regression and the cost function used in SVMs?

Logistic regression uses the log-loss function, while SVMs use the hinge-loss function.

What is the significance of the margin in SVMs?

The margin is the distance between the decision boundary and the closest data points, and maximizing the margin leads to better generalization.

What is the role of the support vectors in SVMs?

Support vectors are the data points that lie on the margin of the decision boundary and determine its shape and position.

How does the choice of the regularization parameter $C$ in SVMs affect the learning process and the resulting classifier?

A larger value of $C$ leads to a more complex decision boundary with a smaller margin, while a smaller value of $C$ leads to a simpler decision boundary with a larger margin.

How do the decision boundaries of logistic regression and SVMs differ?

Logistic regression finds a decision boundary that minimizes the sum of squared errors, while SVMs find a decision boundary that maximizes the margin between the classes.

Logistic regression and SVMs both use hinge loss as their cost function.

False

Support Vector Machines (SVMs) maximize the margin between the decision boundary and the support vectors.

True

In SVMs, the support vectors are the training points that lie outside the margin.

False

The regularizer parameter in SVMs, denoted as C, is inversely proportional to lambda (λ).

True

The alternative view of logistic regression focuses on minimizing the hinge loss function.

False

Support vectors in SVMs solely determine the position of the decision boundary.

False

## Study Notes

### Classification I

- Support Vector Machines (SVMs) are an alternative view of logistic regression.

### Cost Function

- Cost of example: If (want 1), hinge-loss is applied.
- If (want -1), hinge-loss is also applied.

### Support Vector Machine

- SVM:
`min 1/m * Σ [y^(i) * cost1 * θ^T * x_i + (1 - y_i) * cost0 * θ^T * x_i] + λ/2 * Σ θ_j^2`

- By removing 1/m, and multiplying by 1/λ, let
`C = 1/λ`

. - SVM:
`min [Σ y^(i) * cost1 * θ^T * x_i + (1 - y_i) * cost0 * θ^T * x_i] + C/2 * Σ θ_j^2`

### Logistic Regression vs. Support Vector Machine

- Logistic Regression:
`θ^T * x ≤ -1`

or`θ^T * x ≥ 1`

- Support Vector Machine:
`θ^T * x = 0`

### Margin and Support Vectors

- Margin: The distance between the hyperplane and the closest data points.
- Support vectors:
- They are the training points that define the maximum margin of the hyperplane to the data set.
- They determine the shape (position & orientation) of the hyperplane.

- Support vectors properties: They are the points that lie closest to the hyperplane.

Test your understanding of Classification (SVM), Cost Function, Support Vector Machines, and Alternative views of logistic regression from Lecture 3 of the Pattern Recognition course by Dr. Dina Khattab at Faculty of Computer & Information Sciences, Ain Shams University.

## Make Your Own Quizzes and Flashcards

Convert your notes into interactive study material.

Get started for free