Decision Trees Overview and Case Study

Decision trees are a popular supervised classification technique
They are a simple way to make decisions
Decisions represented by hierarchical structures
Questions asked in a hierarchical order
Best trees are short and answer questions with the most relevant information
Decision trees can be built from small datasets and applied to larger populations
Suitable for simple binary decisions

This case study used data mining to predict heart attacks
Data focused on patients with previous heart attacks
Goal to predict which patients were at risk for a second heart attack in the next 30 days
Predicting risk to determine treatment plan
The CART algorithm was employed
More than 100 variables (factors) were included
Data transformation and cleaning occurred before analysis
Factors like age, blood pressure, and sinus problems were considered by the decision tree
The decision tree was 86.5% accurate in predicting heart attack risk

Low blood pressure (<90) strongly suggests a high risk of subsequent heart attacks (70%)
If blood pressure was normal, age was considered as the next factor
Patients under 62 years had almost guaranteed survival (98%)
Older patients' sinus condition was considered if their age was over 62
When sinus was okay, survival chance was 89% ; otherwise, 50% chance
Overall accuracy of the decision tree was 86.5%

Decision tree logic applies to many disease diagnoses
Medical diagnosis involves similar question-and-answer processes
A physician's thought process mirrors decision tree structure when evaluating patient symptoms and ordering tests
Decision trees and decision rules aid in diagnosing diseases
Every question leads to potential answers creating separate branches for further questions
The process continues until a conclusion is reached (leaf node)
Medical professionals and experts in general fields employ similar methods to solve problems

Machine learning uses past data to train and extract knowledge and rules
Decision trees use algorithms to form knowledge from data
Accuracy is measured by the frequency of correct predictions (predictive accuracy)

More training data generally results in better accuracy
More input variables can lead to greater accuracy
A good decision tree is efficient (gets the right answer with the least amount of steps)
Needs a small number of questions

Error measures the decision tree's performance and incorrect predictions
Balance complexity of the tree to avoid overly complex (overfitting) or overly simplified (underfitting) models
Rules in a tree show the logical connections between choices/variables, conditions and predictions. These are the rules leading to branches and nodes

A metric is chosen to evaluate each variable's importance (information gain, Gini impurity, Chi-square)

Choose the variable with the least errors as the root node of the tree
In cases of a tie, choose the variable that has the purest sub-trees
Outlook variable usually gives the best possible root node. Variable chosen is often the best variable when making a decision for the problem
The first question asked becomes "What is the value of Outlook?" to begin the analysis process

The data is divided into subsets based on the root node values (e.g., sunny, overcast, rainy)
Those subsets are further broken down to form sub-trees

Employs divide and conquer method
Steps for building a decision tree:
- Create a root node and assign data
- Identify best splitting variable
- Add branches based on splitting variable values (mutually exclusive)
- Repeat for every leaf node until specified stopping criteria is met

Splitting criteria (which variables to use, how to make bins, for continuous variables)
Stopping criteria (when to stop adding branches)

Trimming the tree to be more balanced and usable (in complex scenarios)
Pruning occurs after construction of the full tree
Tree pruning can help solve imbalances caused by noise, outliers, or overfitting

Decision trees are a popular tool for data mining
They are accurate and easy to use, especially when dealing with limited datasets
Trees are well-suited for communication and explanation of results

Zero errors and 100% accuracy are mostly unrealistic except in specific simple datasets
In real-world data analysis, perfect accuracy is unusual
It is important that the tree is well-balanced
Decision trees help with clear business and practical problems rather than complex ones that defy easy classification