Gradient Descent Optimization

If the loss function $L(w, b)$ is convex, what is the likely impact of different initializations of $w$ and $b$ on the final values obtained after gradient descent?

In the gradient descent algorithm, which of the following statements accurately describes the role of the learning rate, denoted as η?

Consider a scenario where the partial derivative of the loss function $L$ with respect to weight $w$ (i.e., $\frac{\partial L}{\partial w}$) is consistently positive during multiple iterations of gradient descent. What does this indicate?

When updating weights $w$ and bias $b$ using gradient descent, a temporary variable (e.g., temp_w, temp_b) is often used. What issue does using temporary variables prevent?

Assume you are training a model with a very small learning rate. What is the most likely consequence of this choice?

In a multiple linear regression model with K independent variables, how is the predicted value $ŷ$ calculated?

What is the primary objective when adjusting the parameters w and b in a linear regression model?

What does the term 'bias' (b) represent in the context of a linear regression model?

Given a set of training examples $(x_n, y_n)$ from $n=1$ to $N$, and a linear model $ŷ_n = f_{w,b}(x_n) = wx_n + b$, how is the Mean Squared Error (MSE) loss function defined?

In the equation $f_{w,b}(x) = 4x_1 - 2x_2 + 4x_3 + 40$, which term represents the bias?

Given the linear regression model $ŷ = f_{w,b}(x)$ and the loss function $L(w, b)$, which of the following statements best describes the relationship between them?

What does minimizing the Mean Squared Error (MSE) in linear regression achieve?

In the context of linear regression, what do the 'weights' ($w_1, w_2, ..., w_K$) represent?

In the context of gradient descent, what is the expected behavior of the loss function $L(\vec{w}, b)$ if the algorithm is functioning correctly?

What criterion can be used to stop the training process when using gradient descent?

Which of the following statements best describes the difference between model parameters and hyperparameters?

In the gradient descent update rule $w = w - \eta \frac{\partial L}{\partial w}$, if $\frac{\partial L}{\partial w}$ is a negative number, what effect does this have on the value of $w$ in the next iteration, assuming $\eta$ is positive?

For a linear regression model $f_{w,b}(x) = wx + b$, which of the following correctly identifies the inputs/features and the parameters that need to be learned during the training stage?

In the context of gradient descent for a simple linear regression, what does 'convergence' generally imply?

What is the role of the learning rate (η) in the gradient descent algorithm?

In linear regression, what does it indicate if you find parameters $w$ and $b$ such that the loss function $L(w, b)$ is very close to 0 on the training dataset?

Given the MSE loss function $L(w, b) = \frac{1}{2N} \sum_{n=1}^{N} (f_{w,b}(x_n) - y_n)^2$, where $f_{w,b}(x) = wx + b$, what does the term $(f_{w,b}(x_n) - y_n)$ represent?

Which of the following is an example of a hyperparameter in a machine learning model?

How does the number of iterations usually vary across different machine learning tasks when using gradient descent?

How does the gradient descent algorithm update the parameter 'b' (the bias) in a linear regression model?

Consider a scenario where the loss function $L(w, b)$ is nonconvex. What is a potential issue when using gradient descent to minimize this loss function?

In the gradient descent update rules, $\frac{\partial L}{\partial w} = \frac{2}{2N} \sum_{n=1}^{N} (wx_n + b - y_n) \cdot x_n$ and $\frac{\partial L}{\partial b} = \frac{2}{2N} \sum_{n=1}^{N} (wx_n + b - y_n)$, what do these equations represent?

What does the expression wx + b - y represent in the context of calculating the derivatives for linear regression?

Why is it important to ensure that gradient descent is working correctly during the training of a model?

Which of the following is most likely to occur if the learning rate (η) is set too high in gradient descent?

What is the significance of the summation symbol $\sum_{n=1}^{N}$ in the equations for calculating the derivatives of the loss function?

What is the primary purpose of applying a sigmoid function on top of a single output in binary classification?

In the context of binary classification, what does the term 'binary' refer to?

In binary classification, a model outputs a value that is then passed through a sigmoid function. What does this transformed value represent?

For a binary classification problem, if you have two output nodes, which function is typically applied to the outputs to obtain probabilities?

What is the interpretation of the output of a softmax function in a classification problem?

How does Binary Cross-Entropy (BCE) loss function differ from Cross-Entropy loss function based on the information provided?

Consider a scenario where you're building a spam email detector. What would be an appropriate way to assign labels for binary classification?

What is the purpose of the loss function in the context of training a binary classification model?

Which of the following is an appropriate loss function for a binary classification problem using a sigmoid activation function?

In the context of binary classification, if a model predicts a probability of 0.9 for an instance belonging to the positive class, how should this be interpreted?

In logistic regression, what is the purpose of the sigmoid function?

Given the logistic regression equation $\hat{y} = f_{\vec{w},b}(\vec{x}) = g(\vec{w} \cdot \vec{x} + b)$, where $g(z) = \frac{1}{1 + e^{-z}}$, what does $\vec{w} \cdot \vec{x} + b$ represent?

In a logistic regression model predicting whether a student will pass (1) or fail (0) an exam, $x_1$ represents study time and $x_2$ represents exam length. If $f_{\vec{w},b}(\vec{x}) = 0.3$, what is the probability that the student will fail?

What does a large value of $z$ (where $z = \vec{w} \cdot \vec{x} + b$) imply in the context of a logistic regression model?

Why is logistic regression suitable for binary classification problems?

In logistic regression, if the weights $\vec{w}$ are [2, -3] for features $x_1$ and $x_2$ respectively, and the bias $b$ is 1, how does increasing $x_2$ while holding $x_1$ constant affect the predicted probability?

Suppose a logistic regression model predicts the probability of a customer clicking on an ad. If, for a given customer, $z = \vec{w} \cdot \vec{x} + b = 0$, what is the predicted probability of the customer clicking on the ad?

What is the range of possible values for the output of a logistic regression model?

In the context of interpreting logistic regression output, what does $P(y=1)$ represent?

If a student increases their study time ($x_1$) and the coefficient $w_1$ associated with study time in a logistic regression model is positive, what is the likely effect on the probability of passing the exam?

fw,b(x)

Multiple Linear Regression

Weights (w)

Bias (b)

Mean Square Error (MSE)

Loss Function

Adjust w and b

Objective of Linear Regression

Loss Function (L(w, b))

Gradient Descent

Learning Rate (η)

Gradient of L (∇L)

Simultaneous Update

Convergence

Model Parameters

Hyperparameters

Stopping criterion

Iterations

MSE Loss Function

Derivative

Local Minimum

Global Minimum

Binary Classification

Sigmoid Function

Binary Cross Entropy (BCE)

Softmax Function

Softmax Regression

Output Class

Loss Function in Logistic Regression

Optimization Algorithm

Categories in Binary Classification

Learning Objectives of Binary Classification

Logistic Regression

Probability Output

z = w·x + b

Unbounded Value

Pass Probability (ŷ)

Fail Probability

Label (y)

Logistic Function Formula

Interpreting Output