11 Questions
Explain how the variance impacts Ridge and Lasso regression.
The Ridge regression adds penalty equivalent to the square of the magnitude of coefficients, while Lasso regression adds penalty equivalent to the absolute value of the coefficients. This helps in reducing the variance and dealing with overfitting in the models.
Discuss the Naïve Bayes Classifier with an example.
Naïve Bayes Classifier is a probabilistic classifier based on Bayes' theorem with the assumption of independence among predictors. For example, in text classification, it can be used to classify emails as spam or not spam based on the occurrence of certain words.
Explain the concept of R2 score.
R2 score, also known as the coefficient of determination, is a statistical measure that represents the proportion of the variance for the dependent variable that's explained by independent variables in a regression model. It ranges from 0 to 1, where 1 indicates a perfect fit.
Discuss the different types of multiclass strategies in data formats of machine learning.
Some common multiclass strategies include One-vs-All (OvA), One-vs-One (OvO), and Hierarchical classification. OvA treats each class as a binary classification problem, OvO compares each pair of classes, and Hierarchical classification divides the classes into a hierarchy.
Explain how the ROC curve is constructed.
ROC curve is constructed by plotting the True Positive Rate (Sensitivity) against the False Positive Rate (1-Specificity) at various threshold settings. It gives a graphical representation of the trade-off between sensitivity and specificity.
What are the main differences between supervised and unsupervised learnings in machine learning?
Supervised learning involves training on labeled data to make predictions on new data, whereas unsupervised learning involves training on unlabeled data to discover patterns or relationships.
What is overfitting and underfitting in machine learning, and how do they differ?
Overfitting occurs when a model is too complex and performs well on training data but poorly on new data, whereas underfitting occurs when a model is too simple and performs poorly on both training and new data.
What is the concept of Bayes' theorem, and how is it used in machine learning?
Bayes' theorem is a mathematical formula for updating the probability of a hypothesis based on new data, and is used in machine learning for probabilistic modeling and inference.
What is the main difference between classification and regression in machine learning?
Classification involves predicting a categorical label, whereas regression involves predicting a continuous value.
What is stochastic gradient descent, and why is it commonly used in machine learning?
Stochastic gradient descent is an optimization algorithm that updates model parameters based on a single data point at a time, and is commonly used due to its efficiency and scalability.
What is the purpose of cross-validation in machine learning, and how is it used?
Cross-validation is a technique for evaluating machine learning models by dividing the data into training and testing sets, and is used to prevent overfitting and evaluate model performance.
Study Notes
Machine Learning Systems
-
Types of Machine Learning Systems:
- Classic Machine Learning Model: handles fixed data and provides a fixed output
- Adaptive Machine Learning Model: learns from new data and adapts to changes
Supervised and Unsupervised Learning
- Supervised Learning: learns from labeled data to predict output
- Unsupervised Learning: learns from unlabeled data to find patterns or relationships
Reinforcement Learning
- Reinforcement Learning: learns from interactions with the environment to maximize a reward
Challenges of Machine Learning
-
Main Challenges:
- Overfitting
- Underfitting
- Hyperparameter tuning
- Model selection
- Data quality
Hyperparameters
- Hyperparameters: parameters set before training a model, e.g. learning rate, batch size
Bayes' Theorem
- Bayes' Theorem: probability theory that updates the probability of a hypothesis based on new data
- Example: given a disease and a test, calculate the probability of having the disease given a positive test result
Overfitting and Underfitting
- Overfitting: when a model performs well on training data but poorly on new data
- Underfitting: when a model performs poorly on both training and new data
-
Solutions to Overfitting:
- Regularization
- Early stopping
- Data augmentation
Error Measures and Cost Functions
- Error Measures: metrics used to evaluate model performance, e.g. mean squared error, accuracy
- Cost Functions: objective functions to minimize, e.g. mean squared error, cross-entropy
Classification and Regression
- Classification: predicts a categorical output
- Regression: predicts a continuous output
- Binary Classification: predicts one of two classes
- Multiclass Classification: predicts one of multiple classes
Predictive Analysis
-
Types of Predictive Analysis:
- Descriptive analysis: summarizes data
- Predictive analysis: predicts future outcomes
- Prescriptive analysis: recommends actions
Logistic Regression
- Logistic Regression: predicts a probability of a binary output
- Applications: credit risk assessment, customer churn prediction
Model Evaluation Metrics
- Confusion Matrix: summarizes prediction results
- ROC Curve: plots true positive rate vs. false positive rate
- Performance Metrics: accuracy, precision, recall, sensitivity, specificity
Bayes' Theorem Example
- Example: given a disease and a test, calculate the probability of having the disease given a positive test result
Stochastic Gradient Descent
- Stochastic Gradient Descent: optimization algorithm for large datasets
- Need: handles large datasets, reduces computation time
Regularization in Decision Trees
- Regularization Hyperparameters: parameters to control model complexity, e.g. pruning, maximum depth
Cross-Validation
- Cross-Validation: evaluates model performance by splitting data into training and testing sets
- Example: 5-fold cross-validation splits data into 5 folds, trains and tests on each fold
Naïve Bayes Classifier
- Naïve Bayes Classifier: simple probabilistic classifier, assumes independence of features
- Example: given a text and a set of keywords, classify as spam or not spam
R2 Score
- R2 Score: measures the goodness of fit of a regression model
Multiclass Strategies
-
Types of Multiclass Strategies:
- One-vs-All (OVA)
- One-vs-One (OVO)
- Hierarchical classification
ROC Curve
- ROC Curve: plots true positive rate vs. false positive rate
- Construction: plots the ROC curve using true positive rate and false positive rate
Huber Regression
- Huber Regression: robust regression method, reduces the impact of outliers
Polynomial Regression
- Polynomial Regression: predicts a continuous output using a polynomial equation
- Variance: affects the model's performance, e.g. overfitting or underfitting
Principal Component Analysis
- Principal Component Analysis: reduces dimensionality by retaining the most important features
- Applications: image compression, facial recognition
This quiz covers different types of machine learning systems, compares classic and adaptive machine learning models, discusses major applications, compares supervised and unsupervised learning, explains reinforcement learning, challenges in machine learning, hyperparameters, Bayes' theorem, and overfitting.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
