Machine Learning in Medicine

Lecture 1: Introduction

• Machine Learning (ML) automates large-scale medical data analysis to extract patterns and insights
• Traditional rule-based approaches struggle to discern those patterns
• ML assists in diagnostics, risk prediction, and treatment recommendations
• Diagnostic errors may be reduced, these contributing to around 10% of patient deaths and hospital adverse events
• Supervised Learning: models trained on labeled data
• Unsupervised Learning: models identifying patterns in unlabeled data for classification
• Reinforcement Learning: models learning through rewards and penalties
• Robust algorithms are crucial given medical data's complexity, noise, and variability
• Reliability is ensured, minimizing errors in diagnostics and predictions
• Remote health monitoring uses algorithms to handle diverse sensor outputs accurately
• Many physiological time series are recorded, these often unused clinically
• ML bridges the gap by extracting meaningful patterns from raw data
• Clinicians are assisted in data-driven decisions, reducing diagnostic errors and improving healthcare efficiency
• Traditional pattern recognition relies on predefined rules, methods which poorly generalize to complex medical data
• ML, particularly deep learning, extracts intricate patterns from large datasets
• Deep learning improves medical imaging analysis and long-term physiological monitoring
• Rule-based systems need manually defined rules, may becoming inflexible and failing in complex scenarios
• ML learns patterns from data directly allowing adaptability to new cases in medical diagnostics
• Reinforcement Learning (RL) optimizes decision-making through trial-and-error learning
• RL optimizes treatment plans, adjusting medication dosages, and improving robotic surgery techniques
• RL improves these techniques by continuously learning from patient responses
• AlphaFold predicts protein structures with high accuracy
• AlphaFold addresses a fundamental problem in biology
• Understanding protein structures is crucial for drug discovery and disease research
• Misfolded proteins are associated with diseases like Alzheimer's and Parkinson's

Lecture 2: Data Preprocessing

• Data preprocessing is crucial because "Garbage in = Garbage out" as poor-quality data leads to inaccurate predictions
• Preprocessing ensures data quality by handling missing values, normalizing scales, and identifying outliers
• Data preprocessing steps:
  • Load to understand data types
  • Inspect by performing exploratory data analysis
  • Clean to handle missing values, outliers, and errors
  • Rescale by normalizing or standardizing data
• Data types used in Machine Learning:
  • Numerical (double/int): continuous or discrete values
  • Boolean: true/false values
  • Categorical: fixed number of possible values, like colors
  • Ordinal: categorical with a natural order, like education level
• Exploratory data analysis (EDA) helps gain insights into datasets
• EDA identifies patterns, anomalies, and relationships between variables using statistical and visual techniques
  • Histograms
  • Scatter plots
  • Boxplots
• Common visualization techniques for data exploration:
  • Line plots for time series data
  • Bar charts for category-based comparisons
  • Histograms summarizing data distribution
  • Boxplots highlighting median, quartiles, and outliers
  • Scatter plots showing relationships between two variables
• Methods to handle missing data:
  • Removing missing values by dropping features or samples
  • Imputation replaces missing values with mean, median, or mode
  • K-Nearest Neighbors (KNN) imputation uses similar data points to estimate missing values
• Outliers can be detected using:
  • Distance-based methods identifying points far from others (k-nearest neighbors)
  • Density-based methods checking local density differences
  • Anomaly detection techniques, use of Local Outlier Factor (LOF)
• Ways to handle outliers:
  • Removing extreme outliers
  • Re-weighting them to reduce their influence
• Standardization (Z-score normalization): rescales data to have a mean of 0 and standard deviation of 1
• Normalization (Min-Max scaling): rescales data to a fixed range, typically [0,1] or [-1,1]
• Normalization is more sensitive to outliers
• Standardization is preferred for algorithms requiring normally distributed data
• The "Five Number Summary Statistics" consists of:
  • Min
  • Q1 (25th percentile)
  • Median (50th percentile)
  • Q3 (75th percentile)
  • Max
• Provide a quick overview of data distribution and can be visualized effectively using boxplots
• Unverified or poorly understood data introduces biases, leads to incorrect conclusions, and reduces model reliability
• Ensuring clean, well-labeled, and well-understood data is a fundamental part of building effective ML models

Lecture 3: Linear Regression

• The goal of linear regression is to predict an independent variable y based on one or more dependent variables X
• Linear regression estimates relationships among continuous variables by assuming a linear function of the form: y=w0+w1x1+w2x2+...+wnxn
• ww representing the model’s coefficients.
• Coefficients W are estimated by minimizing the Mean Square Error (MSE): J(w) = ∑(y – Χω)2
• Minimization can be solved with matrix inversion, gradient descent, or the normal equation
• Coefficients can be estimated using matrix inversion (if possible)
• Gradient descent is used if inversion is not feasible
• The Normal Equation is a closed-form solution to finding the optimal weights ww in linear regression: w = (XTX)⁻¹XT y
• Normal Equation directly computes the optimal coefficients without requiring iteration
• Normal Equation is computationally expensive when X is large
• Linear regression assumes:
  • Linearity: the relationship between input X and output y is linear
  • Independence: observations are independent of each other
  • Homoscedasticity: the variance of the residuals is constant
  • Normality of Errors: the noise in y follows a normal distribution
• Data must follow a normal distribution: y = f(x, w) + є, € ~ Ν(0, σ²)
• Algebraic Approach: finds w by minimizing the mean squared error (MSE)
• Probabilistic Approach: assumes that y follows a Gaussian distribution around f(X)
• Matrix must be invertible for normal equation work
• Linear inversion can be impossible if XTX is not full-rank and columns are linearly dependent
• Using the pseudo-inverse or regularization techniques happens when X has more features than samples, XTX becoming singular
• Linear regression extends to multiple features (multivariate regression) using matrix notation:y=Xw
• X is the design matrix containing multiple features
• w a vector of coefficients
• Linear regression solution remains the same with need for higher-dimensional optimization
• The MLE approach assumes that the noise in the data follows a Gaussian distribution: y = f(x, w) + €, ε~ Ν(0, σ²)
• Maximizing the log-likelihood function minimizes squared errors, what least squares do
• Both methods yield the same estimator for w with the probability attitude also estimating the noise by calculating a derivative according to beta
• Regression predicts continuous values (e.g., how long will a patient stay in the ICU?)
• Classification predicts discrete labels (e.g., will the patient survive the ICU stay, specifically yes/no?)
• The method chosen depends on the nature of the output variable

Lecture 4: Linear Models for Classification

• Regression estimates relationships among continuous variables
• Classification identifies decision boundaries between classes like with malignant or benign tumors
• The logistic regression hypothesis function is: h(x) = g(wTx) = σ(wTx)
• The sigmoid function is σ(z) = 1 / (1 + e -z)
• The output of logistic regression is interpreted as a probability
• h(x) = P(y=1|x, w) means that h(x) = 0.7 indicates a 70% chance of belonging to the positive class
• The cost function in logistic regression is the cross-entropy cost function
• The cross-entropy formula is: J(w) = ∑[y(i) log(h(x(i))) + (1 – y(i)) log(1 – h(x(i)))]
• Cross-entropy is a convex cost function which gradient-based optimization methods converge to the global minimum
• Cross-entropy corresponds to the maximum likelihood estimate
• Optimize parameters using gradient descent while updating parameters: wj := wj – α ∑(h(x(i)) – y(i))x(i)j
• Feature scaling helps gradient descent converge faster in logistic regression
• For unscaled features the convergence can lead to slow or unstable training
• Logistic regression extends to multiclass classification using:
• One-vs-all (one-vs-rest) transforming the problem into binary classification problems
• Multinomial logistic regression (SoftMax regression) generalizing logistic regress with the SoftMax function
• The SoftMax function Aj normalizes an input vector into a probability distribution across multiple classses