18 Questions
What is the purpose of gradient ascent in maximizing the log-likelihood function?
To update the parameters in the direction of the gradient
What is the update rule for gradient descent?
w ← w - α∇w f(w)
What is the role of the step size α in gradient ascent?
It determines the learning rate of the algorithm
What is the purpose of computing the gradient vector?
To obtain the local direction of steepest ascent
What is the goal of using gradient descent in training a neural network?
To minimize the loss function of the model
What is the significance of the gradient vector in gradient descent?
It gives the local direction of steepest descent
What is the purpose of the softmax function in the given context?
To perform normalization to output a probability distribution
What does the expression m(w) represent?
The likelihood of a particular set of weights explaining the observed labels and datapoints
Why is the log-likelihood expression used instead of the likelihood expression?
Because log is an increasing function
What is the difference between a multi-layer perceptron and a multi-layer feedforward neural network?
The type of non-linearity applied
What is the goal of optimizing the weights of a neural network?
To maximize the likelihood of the observed data
What is the advantage of using the log-likelihood expression in mini-batched or stochastic gradient descent?
It is more stable
What is the goal of running gradient ascent on the function m(w)?
To maximize the likelihood of the true class probabilities
What is the main drawback of using batch gradient descent?
It is too slow
What is the purpose of mini-batching?
To speed up gradient descent
What is the limit of mini-batching where the batch size k = 1?
Stochastic gradient descent (SGD)
What is the relation between the number of datapoints in the batch and the computation of gradients?
The number of datapoints in the batch decreases the computation of gradients
What is the goal of updating the parameters w?
To reach a local minimum of the function
Learn about gradient ascent and descent, a key concept in machine learning used to maximize log-likelihood functions. Understand how to calculate the gradient vector and update parameters along the direction of the gradient. Test your knowledge of this fundamental algorithm in machine learning.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
