38 Questions
What is the purpose of an optimization algorithm in machine learning?
To find the local minima of estimated errors
What type of regression is used to predict a categorical dependent variable?
Logistic Regression
What is the name of the iterative first-order optimization algorithm used to find a local minimum/maximum of a given function?
Gradient Descent
What is the purpose of Multiple Linear Regression?
To estimate the relationship between a quantitative dependent variable and two or more independent variables
What is an example of an application of Multiple Linear Regression?
House prices prediction by given different attributes
What is the purpose of Logistic Regression?
To predict a categorical dependent variable using a given set of independent features
What is the main goal of regression analysis?
To predict the value of a dependent variable based on the value of at least one independent variable
What is the primary objective of simple linear regression?
To model the relationship between one independent variable and a dependent variable
What is the type of machine learning task addressed by regression analysis?
Regression
What is the term for the variable being predicted or explained in regression analysis?
Dependent variable
What is the purpose of the error term in simple linear regression?
To account for the random variation in the data
What is an example of a regression task?
Weather forecasting
What is the equation for a simple linear regression model?
Y = β0 + β1X + ε
What is the term for the variable used to predict or explain the dependent variable?
Independent variable
What is the estimated mean value of Y when the value of X is zero?
β0
What is the goal of ML regression?
To find the best coefficients to predicate the actual value
What is the purpose of the least square method?
To minimize the sum of the squared differences
What does β1 represent in the regression equation?
The estimated change in the mean value of Y as a result of a one-unit increase in X
What is the estimate of the population regression line?
Ŷi = b0 + b1Xi
What is the purpose of finding the values of b0 and b1?
To minimize the sum of the squared differences
What does the predicted value of Y represent?
The estimated value of Y for Xi
What is the equation for finding the values of b0 and b1?
min ∑ (Yi - Ŷi) = min ∑ (Yi - (b0 + b1Xi))
What is the purpose of modifying the cost/error minimization function in locally weighted regression?
To derive a local approximation of the target function
What is the role of the query point 𝑥𝑞 in the error criterion 𝐸(𝑥𝑞)?
It is the point at which the error is being evaluated
What is the advantage of criterion three over criterion two?
It has a computational cost independent of the total number of training examples
What is the form of the linear function used to approximate the target function 𝑓 near 𝑥?
𝑓(𝑥) = 𝑤0 + 𝑤1 𝑎1 𝑥 + 𝑤2 𝑎2 𝑥 + ⋯ + 𝑤𝑛 𝑎𝑛 𝑥
Why is criterion two not preferred despite being aesthetically pleasing?
It is computationally expensive and grows linearly with the number of training examples
What is the purpose of the weighting function K in criterion two?
To weight the error of each training example by its distance from 𝑥𝑞
What is the advantage of using the squared error in locally weighted regression?
It is a simple and intuitive measure of error
What is the role of the number k of nearest neighbors in criterion three?
It is the number of nearest neighbors used to compute the error
What does the equation 𝒉𝜽 = 𝝈(𝜽𝑻 𝒙) represent in the context of logistic regression?
The probability of the dependent variable being 1
What is the purpose of the intercept term 𝜽𝟎 in the logistic regression model?
To account for the intercept or constant term in the model
What is the maximum likelihood estimation method used for in logistic regression?
To find the optimal parameters for the logistic regression model
What is the difference between the maximum likelihood estimation method used in logistic regression and the least square estimation method used in linear regression?
The maximum likelihood estimation method is used for logistic regression and the least square estimation method is used for linear regression
What is the form of the logistic function 𝝈(𝒛) in the logistic regression model?
𝝈(𝒛) = 1 / (1 + 𝑒^(-𝒛))
What is the significance of the term 𝒙𝟎 in the logistic regression model?
It is always equal to 1
What is the relationship between the input features 𝒙 and the coefficients 𝜽 in the logistic regression model?
𝜽 and 𝒙 are linearly related
What is the purpose of logistic regression in machine learning?
To predict a categorical dependent variable
Study Notes
Regression Analysis
- Aims to solve a Machine Learning problem by approximating a real-valued function 𝑓 𝑥 ∈ ℝ (i.e., predicting a continuous output)
- Examples of regression tasks: house prices prediction, weather forecasting, stocks market prediction, spread of viral disease, estimating salaries
Simple Linear Regression Model
- Only one independent variable, X
- Relationship between X and Y is described by a linear function 𝒇(𝒙) = 𝒚
- Changes in Y are assumed to be related to changes in X
- Equation: Yi = β0 + β1Xi + ε i
- Intercept = β0, Slope = β1
Predication Line in Regression Equation
- Estimated regression line provides an estimate of the population regression line
- b0 is the estimated mean value of Y when the value of X is zero
- b1 is the estimated change in the mean value of Y as a result of a one-unit increase in X
The Least Square Method
- b0 and b1 are obtained by finding the values that minimize the sum of the squared differences between 𝑌 (real value) and 𝑌 (estimated value)
Finding Best Weights/Coefficients
- Goal is to find the best coefficients to predicate the actual value
- Optimization algorithm is applied to find the best values of these coefficients by finding the local minima of estimated errors
- Example of optimization algorithm: Gradient Descent (GD)
Regression Models: Other Types
- Multiple Linear Regression: estimates the relationship between a quantitative dependent variable and two or more independent variables
- Logistic Regression: used to predict dependent variable (as categorical or class) using a given set of independent feature
Multiple Linear Regression
- Hypothesis function of linear regression: 𝑓(𝑥) = 𝑤0 + 𝑤1 𝑎1 𝑥 + 𝑤2 𝑎2 𝑥 + ⋯ + 𝑤𝑛 𝑎𝑛 𝑥
Locally Weighted Regression
- Modify the error criterion E to emphasize fitting the local training examples
- Minimize the squared error over just the k nearest neighbors
- Combine 1 and 2 for a good approximation
Logistic Regression
- Used to predict dependent variable (as categorical or class) using a given set of independent feature
- Equation: 𝒉𝜽 = 𝝈(𝜽𝑻 𝒙)
- Model: 𝝈 𝒛 = 1+𝑒 −𝒛
Optimizing Logistic Regression
- Use Maximum likelihood estimation (MLE) method for estimation of accuracy (finding best parameters/weights for the logistic regression model)
This quiz covers linear and logistic regression concepts in machine learning, including simple and multiple linear regression, locally weighted regression, and logistic regression for classification. It is part of the CS364 course in Machine Learning.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
