Questions and Answers
What is the purpose of model selection?
To ensure that the model is not overly complex
What is the purpose of k-fold cross-validation?
To divide the data into k subsets
Which of the following is a measure of error for regression models?
RMSE
What is an example of overfitting?
Signup and view all the answers
What is an example of underfitting?
Signup and view all the answers
Study Notes
- Evaluating model performance in data science is important to avoid overfitting.
- There are two methods of evaluating models in data science: hold-out and cross-validation.
- In k-fold cross-validation, we divide the data into k subsets of equal size.
- We build models k times, each time leaving out one of the subsets from training and use it as the test set.
- Training errors: Errors committed on the training set
- Test errors: Errors committed on the test set
- Generalization errors: Expected error of a model over random selection of records from same distribution
- Underfitting: when model is too simple, both training and test errors are large
- Overfitting: when model is too complex, training error is small but test error is large
- Reasons for model overfitting: not enough training data, over training (train NN with large number of iterations).
- Model selection is performed during model building.
- The purpose of model selection is to ensure that the model is not overly complex and that generalization error is estimated.
- Two types of evaluation are used in data science: classification evaluation and regression evaluation.
- Classification evaluation uses a confusion matrix to measure the number of correct and incorrect predictions made by the classification model.
- Regression evaluation uses performance metrics such as accuracy and precision.
- The key factors to consider when evaluating a regression model are its sensitivity, specificity, and RMSE.
- The RMSE is a popular measure of error for regression models, but it can only be compared between models whose errors are measured in the same units.
- The RSE can be compared between models whose errors are measured in the different units, while the MAE is usually similar in magnitude to the RMSE but smaller.
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
