Deep Learning in NLP: Activation Functions and Neural Network Architectures

0.332

0.1

To control the step size of each update of the weights and biases

<p>To introduce non-linearity in the hidden layer</p> Signup and view all the answers

<p>To minimize the error between the predicted output and the actual output</p> Signup and view all the answers

<p>At least one iteration</p> Signup and view all the answers

<p>To fine-tune the weights of a neural net based on the error rate obtained in the previous epoch.</p> Signup and view all the answers

<p>The number of layers.</p> Signup and view all the answers

<p>$y = 1/(1 + e^{-x})$</p> Signup and view all the answers

<p>To determine the loss and fine-tune the weights of the neural net.</p> Signup and view all the answers

<p>$y = max(ax, x)$</p> Signup and view all the answers

<p>To minimize the error rate and increase the generalization of the model.</p> Signup and view all the answers

<p>To compute the error gradients and update the weights and biases to minimize the error during training</p> Signup and view all the answers

<p>A multilayer feed-forward neural network, and its characteristic is that the information flows only in one direction, from input layer to output layer, without any feedback loops</p> Signup and view all the answers

<p>The sigmoid function, and its formula is y = 1 / (1 + e^(-x))</p> Signup and view all the answers

<p>The error is calculated as the difference between the predicted output and the actual output, and then propagated backwards to update the weights and biases</p> Signup and view all the answers

<p>The learning rate is 0.9, and its role is to control the step size of each weight update during the training process</p> Signup and view all the answers

<p>The bias term is used to shift the activation function, and it is updated during training using the error gradients and the learning rate</p> Signup and view all the answers