Introduction to Linear Regression

A model where output directly corresponds to input, often represented by a straight line.

The relationship between input and output cannot be described as a simple linear function.

A simple linear equation used to predict a continuous outcome (like potato yield) based on a single feature (like fertilizer amount).

The value we want to predict, like potato yield, in simple linear regression.

The feature used to make predictions, such as fertilizer amount in simple linear regression.

The starting point of a simple linear regression line. It's the value of Y when x1 equals zero.

The rate of change in a simple linear regression line. It tells us how much Y changes for every one-unit increase in x1.

A vector of parameters (w) and a real number (b) that define a linear model to predict a target value based on features.

A mathematical equation that describes the relationship between input features and an output target value. It's often represented as a straight line in a 2D plot, or a plane/hyperplane in higher dimensions.

The process of finding the best parameters (weights and bias) for a regression model. It involves minimizing a loss function that measures how well the model fits the training data.

A measure of how poorly a model fits the data. It penalizes the model for making incorrect predictions.

A specific type of loss function commonly used in linear regression, which measures the squared difference between predicted and actual values.

A function that represents the loss function averaged over the entire training dataset. It's minimized during optimization to find the best model parameters.

A direct formula to find the optimal parameters of a model without iterative optimization. It's simpler but might not always be applicable.

An iterative optimization algorithm that repeatedly adjusts model parameters to minimize a cost function. It starts with an initial guess and gradually updates the parameters until a minimum is reached.

The direction of most rapid change in a function. In gradient descent, it indicates the direction to adjust parameters to decrease the cost function.

The "learning rate" is the size of the step taken during each iteration of gradient descent, controlling how quickly the algorithm adjusts towards the minimum of the cost function. A larger learning rate leads to bigger jumps, potentially converging faster but risking overshooting the optimal point. A smaller learning rate results in smaller steps, ensuring a smoother descent but requiring more iterations.

The "gradient" in gradient descent refers to the direction of the steepest increase in the cost function. Gradient descent works by moving in the opposite direction of the gradient, which leads towards the minimum point where the cost function is lowest.

Gradient Descent is an iterative algorithm used for finding the minimum value of a cost function. It works by repeatedly adjusting model parameters in the direction of the steepest descent, aiming to find the parameter values that minimize the error between predicted and actual values.

Gradient Descent converges when the model's parameters reach a point where further adjustments do not significantly reduce the cost function. This indicates that the algorithm has found a local minimum, although it may not necessarily be the global minimum.

Mean Squared Error (MSE) is a common cost function used in linear regression. It measures the average squared difference between the predicted values and the actual values. By minimizing MSE, the algorithm aims to find the line that best fits the data points.