Machine Learning 1 - Week 2 Lecture Quiz

Machine Learning 1 - Week 2 Lecture

• Supervised machine learning was the focus of the lecture.

Categories of Analytics

• Analytics are categorized into different types:

  • Descriptive: Describes past performance (e.g., history, trends).
  • Diagnostic: Explains causes of trends.
  • Predictive: Forecasts future trends.
  • Prescriptive: Recommends actions.

• Example questions for each category, regarding customer churn:

  • Descriptive: Which customer churned?
  • Diagnostic: Why did the customer churn?
  • Predictive: Which customers will churn?
  • Prescriptive: What can I do to change the outcome of customer churn?

The Bias-Variance Tradeoff

• Minimizing errors involves minimizing both bias and variance.

• Variance is always non-negative; bias can be negative.

• It's easier to optimize for one type of error, but optimizing for both (bias and variance) provides the best outcome.

• A graph was shown of Mean Squared Error vs. Flexibility, outlining the tradeoff.

• Some models are shown along the diagram of different types of algorithm.

Statistical Learning

• The error in prediction is a combination of reducible and irreducible errors.
• Reducible error is due to the model's ability to learn patterns from data.
• Irreducible error is due to factors outside the model, such as inherent randomness.

Statistical vs. Machine Learning Thinking

• Statistical thinking focuses on inference, uncertainty quantification, and assumptions validation.
• Machine learning focuses on finding patterns in data to create accurate predictions on new, unseen data.
• This can be done using an experimental approach, trying different algorithms and methods.

Machine Learning vs. Statistical Thinking (Concepts)

• Simplicity: Occam's razor – use the simplest model that works. Sometimes, increased complexity is needed.
• Divergence: Data pre-processing, transformation, and model validation differs between statistical methods and machine learning approaches/methods.
• Outliers: Machine learning focuses on how outliers affect predictions, while statistics focuses on interpretation.
• Feature Selection: Machine learning selects features based on their impact on predictions, while statistics focuses on interpreting relationships.
• Results: Machine learning emphasizes prediction accuracy, while statistics emphasizes inference and confidence intervals.

Structure of Training and Prediction

• Data is split into training and testing sets.
• A model is created with parameters.
• The model is trained using the training data.
• Prediction is made on the testing data, and accuracy is measured using the testing data set.

Threshold

• Many models output probabilities.
• A threshold is used to classify an observation based on the probability (e.g., above 0.5 predict as 1, below 0.5 predict as 0).
• Experimenting with changing thresholds is helpful.
• Choosing a threshold depends on considerations of importance of the different types of errors (false positives and false negatives).

ROC Curve

• ROC (Receiver Operating Characteristic) curves plot True Positive Rate (TPR) against False Positive Rate (FPR).
• ROC curves show how TPR and FPR change as the threshold changes.
• Area under the ROC curve (AUC) can serve as a measure of classifier performance, especially valuable with imbalanced datasets.
• ROC curves can help determine the optimal threshold to minimize the costs of false predictions (false positive and false negative).