Untitled Quiz

A loss function used in binary classification to measure the difference between predicted probabilities and actual labels.

Transforms a vector of values into a probability distribution.

Loss function used to evaluate performance for multiple target classes.

Percentage of correctly classified instances. (Correct Predictions / Total Predictions)

Percentage of actual positive instances correctly identified.

Percentage of predicted positive instances that are actually positive.

Activation function that outputs a value between 0 and 1. Common for binary classification.

Activation function that outputs the input if it's positive, otherwise 0. Common choice.

A probabilistic classifier that assumes features are independent.

Transforms the weighted sum of inputs to a neuron into an output.

In KNN regression, when k > 1, the prediction for a new data point is the average value of its k nearest neighbors.

KNN uses distance functions to measure the similarity between data points. Common functions include Euclidean distance, Manhattan distance, Minkowski distance, and Hamming distance.

The process of finding the optimal value for a parameter that is not learned during training, such as k in KNN.

To prevent bias towards features with larger scales, normalization transforms features to a common range.

A model that uses a linear equation to predict a continuous outcome based on one or more input features. Represented as ŷ = θ1 x1 + θ2 x2 + … + θd xd + θ0.

The parameters of a linear regression model are θ1, θ2, ..., θd, and θ0. θ1 represents the slope, and θ0 represents the intercept.

The goal of linear regression is to find the optimal values for the parameters that minimize the difference between predicted values and actual values.

Linear regression can be extended to handle multiple features, resulting in a plane (2 features) or a hyperplane (more than 2 features).

Linear regression can be expressed in a concise vectorized form: ŷ = θ T x, where θ is a vector of parameters and x is a vector of features.

The error term (ε) in linear regression represents the difference between the predicted value and the actual value, which is usually assumed to be normally distributed.

The process of feeding input data to a neural network and calculating the predicted output.

Parameters in a neural network that determine the strength of connections between neurons.

A constant value added to the weighted sum of inputs at each neuron.

The weighted sum of inputs before applying the activation function.

The output of the activation function, representing the neuron's firing strength.

The final layer of the neural network, responsible for producing the prediction.

A function that measures the difference between the predicted output and the actual output.

The output from a linear regression model before it's passed through the sigmoid function. It represents the unnormalized score.

The output of the sigmoid function, representing the probability that the input belongs to the positive class.

A value used to determine the class label based on the logistic regression output.

The process of finding the optimal parameters that minimize the loss function, resulting in improved classification accuracy.

A line or surface in the feature space that separates points predicted as positive from those predicted as negative.

An extension of logistic regression that handles multiple classes, using the softmax function to output probabilities for each class.

A table summarizing the performance of a classification model by showing the counts of true positives, true negatives, false positives, and false negatives.

The proportion of correctly classified instances out of all instances.

When training and testing data come from different distributions, leading to poor model performance in real-world applications.

Dividing data into three sets: training (model learning), validation (hyperparameter tuning), and testing (final performance evaluation).

Creating new synthetic data from existing data to increase the dataset size, especially helpful when data is limited.

Comparing the data used during model training to the data encountered in real-world deployment to identify discrepancies and potential causes of performance issues.

Manually analyzing and categorizing errors in model predictions to identify patterns and pinpoint specific areas for improvement.

Starting with a broad range of hyperparameter values and narrowing down the search space gradually to find the optimal settings.

Adjusting the size of hidden layers in a neural network with linear increments.

Evaluating model performance separately on validation and testing data. Validation guides hyperparameter tuning; testing measures final performance.

Data encountered by the model in real applications, potentially differing from the training and testing data.

Adjusting the weights of a neural network to minimize the loss function. Just like in linear and logistic regression, we aim to find the best weights to improve predictions.

Using derivatives to calculate the effect of each weight on the loss. This helps understand how changing each weight impacts the overall prediction accuracy.

Calculates derivatives of complex functions by breaking them down into simpler parts. We use this rule to compute how changes in one layer affect the output and ultimately the loss.

Start with the derivative of the output (ŷ) with respect to the activation function (a) and work your way backward through the network.

Calculates the derivative of the activation function (like sigmoid) with respect to the input (z). This determines how much the activation function contributes to the loss.

Calculates the derivative of the weighted sum (z) with respect to each weight (W). This helps us see how each weight affects the input to the activation function.

Calculates the derivative of the weighted sum (z) with respect to the bias term (b). This measures the bias's impact on the input to the activation function.

Calculates the derivative of the weighted sum (z) with respect to the activation function (a) of the previous layer. This helps understand the influence of the preceding layer on the current layer's output.

Calculating the derivatives of all weights and biases in a single operation. A more efficient way to update the network parameters.

The derivative of the prediction (ŷ) with respect to the activation function (a). It's usually set to 1, as the prediction directly depends on the activation function.

Derivatives provide crucial information about the network's parameters. They allow us to adjust weights and biases to minimize the loss function and improve prediction accuracy.

Backpropagation is the process of using derivatives to calculate how changes in weights and biases affect the output of a neural network. We use this information to adjust the parameters to reduce the loss function and improve the model's performance.

To update the weights and biases of a neural network to minimize the difference between the predicted outputs and the actual desired outputs.

The values of weights and biases determine how the input signals are combined and transformed within a neural network, leading to a final prediction.

By using derivatives, backpropagation helps us calculate the effect of each parameter on the loss function. This allows us to adjust the weights and biases in the right direction to minimize the loss and improve the model's accuracy.