40 Questions
What is Error composed of?
classification error + margin error
In the perceptron algorithm, how do we approach misclassified points?
by walking the line towards them
What is the value of yi for points in the plus class?
1
What is the condition for data points in the plus class?
w.xi + b ≥ 1
What can the perceptron algorithm be viewed as?
an algorithm that minimizes an error function
What is the function of 𝒇 in Linear Classifiers?
to return the sign of (𝒘.𝒙 + 𝑏)
What is the purpose of the margin in the Plus-Plane?
To maximize the distance between classes
What does 𝒘 denote in Linear Classifiers?
the weight vector
What is the equation for the decision boundary?
w.x + b = 0
What is the sign of 𝒘.𝒙 + 𝑏 when the output is +1?
positive
What is the role of the bias term b in the equation?
To shift the decision boundary
What is the relationship between the Plus-Plane and the Minus-Plane?
They are parallel planes
What is the purpose of the perceptron algorithm?
to classify data points correctly
Where is the data classified when 𝒘.𝒙 + 𝑏 is positive?
as +1
What is the condition for a point to be in the minus class?
xi + b ≤ -1
What is the condition for data points in the minus class?
w.xi + b ≤ -1
What is the value of b in the given equation?
b = 1
What is the purpose of the decision boundary?
To separate the classes
What is the formula for the margin width M?
M = 1/(2s)
What is the equation for the Plus-plane?
w.x + b = 1
What is the equation for the Minus-plane?
w.x + b = -1
Why is the vector w perpendicular to the Plus Plane?
It is not stated in the provided text
What is the relationship between xi and yi if yi = -1?
xi + b = -1
What is the purpose of computing the margin width M?
It is not stated in the provided text
What is the relationship between the vector w and the Plus Plane?
w is perpendicular to the Plus Plane
What is the equation for the Plus Plane?
w.x + b = 1
What is the equation for the Minus Plane?
w.x + b = -1
What is the relationship between x+ and x-?
x+ is the closest Plus Plane point to x-
What is the claim about the relationship between x+ and x-?
x+ = x- + l w for some value of l
What is the margin width M?
M = 1 / (w.x + b)
What is the equation for the point x+ in terms of x- and w?
x+ = x- + l w
What is the value of w.(x - + l w) + b?
1
What is the primary function of the C parameter in a Support Vector Machine?
To control the trade-off between the slack variable penalty and the width of the margin
What is the effect of a small C parameter on the margin of a Support Vector Machine?
It leads to a large margin
What is the purpose of the kernel trick in Support Vector Machines?
To map a higher-dimensional space to a lower-dimensional space
What is the characteristic of a Radial Basis Function (RBF) kernel?
It is only defined by the relative position of the data points
What is the role of λ in the cost function of a Support Vector Machine?
It controls the trade-off between the fit term and the regularization term
What is the result of replacing the raw input variables with a much larger set of features in a Support Vector Machine?
A planar separator in the high-dimensional space
What is the relationship between C and λ in a Support Vector Machine?
C is inversely proportional to λ
What is the advantage of using the kernel trick in Support Vector Machines?
It allows for non-linear separators in the original feature space
Study Notes
Error and Perceptron Algorithm
- Error = classification error + margin error
- The perceptron algorithm minimizes an error function
- The algorithm can be seen as an iterative process that adjusts a random line to correctly classify misclassified points
Linear Classifiers
- A linear classifier is defined as f(x, w, b) = sign(w.x + b)
- w denotes the weights, x denotes the input, and b denotes the bias
- The classifier outputs +1 or -1 depending on the sign of w.x + b
Conditions for Optimal Separating Hyperplane
- w.xi + b ≥ 1 if yi = 1 (points in plus class)
- w.xi + b ≤ -1 if yi = -1 (points in minus class)
Computing Margin Width
- Margin width (M) can be computed using w and b
- The vector w is perpendicular to the Plus Plane
- The Plus-plane = {x : w.x + b = +1} and Minus-plane = {x : w.x + b = -1}
- Margin width is the distance between the Plus-plane and Minus-plane
Support Vector Machine – C Parameter
- C is the regularization parameter that controls the trade-off between the slack variable penalty and width of the margin
- Small C makes the constraints easy to ignore, leading to a large margin
- Large C allows the constraints to be hard, leading to a small margin
Linear Separability – Kernel Trick
- Mapping input data to a higher dimension can make a non-linearly separable problem linearly separable
- The kernel trick allows for efficient computation in high-dimensional spaces
- Radial Basis Function (RBF) kernel is a type of kernel that only depends on the distance from a center point
Learn about the Perceptron algorithm, its connection to Neural Networks, and how it minimizes error functions to classify points correctly.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
