## Questions and Answers

What is the primary goal of the greedy decision tree learning algorithm?

To select the best feature to split the data

What is the problem of selecting the best feature to split the data referred to as?

Problem 1: Feature Selection

How is the classification error calculated in a decision tree?

By calculating the proportion of misclassified instances

What is the purpose of calculating the entropy in a decision tree?

Signup and view all the answers

What is the stopping condition in a decision tree?

Signup and view all the answers

What is the main difference between the greedy decision tree learning algorithm and other decision tree algorithms?

Signup and view all the answers

What is the problem of determining when to stop splitting the data referred to as?

Signup and view all the answers

What is the entropy when all examples are of the same class?

Signup and view all the answers

What is the purpose of calculating the information gain in a decision tree?

Signup and view all the answers

What is the formula for calculating entropy?

Signup and view all the answers

What is the information gain in the context of decision trees?

Signup and view all the answers

What is the purpose of the feature split selection algorithm?

Signup and view all the answers

What is the entropy of the example 'ssss'?

Signup and view all the answers

What is the information gain when splitting on the 'Credit' feature?

Signup and view all the answers

Why is the decision split on the 'Credit' feature?

Signup and view all the answers

What is the entropy of the example 'sf'?

Signup and view all the answers

What is the primary objective when choosing feature h*(x) in decision tree learning?

Signup and view all the answers

What is the next step after learning a decision stump?

Signup and view all the answers

What is the first stopping condition in decision tree learning?

Signup and view all the answers

What is the main characteristic of greedy decision tree learning?

Signup and view all the answers

What is the final output of the decision tree prediction algorithm?

Signup and view all the answers

What is the main idea behind tree learning?

Signup and view all the answers

## Study Notes

### Decision Trees

- Step-by-step process for greedy decision tree learning:
- Start with an empty tree
- Select a feature to split data
- For each split of the tree, determine whether to make predictions or continue splitting

- Feature selection criteria:
- Entropy and Information Gain

### Entropy and Information Gain

- Entropy: measures the impurity of data
- If all examples are of the same class, entropy = 0
- If examples are evenly split between classes, entropy = 1
- Formula: 𝐸𝑛𝑡𝑟𝑜𝑝𝑦 = ∑ - 𝑝𝑖 log 2 𝑝𝑖

- Information Gain: measures the effectiveness of a split
- Formula: Information Gain = Entropy (parent) - (weighted average) Entropy (children)

### Calculating Information Gain

- Example 1: Credit feature
- Entropy (parent) = 0.9905
- Entropy (excellent) = 0
- Entropy (fair) = 0.8865
- Entropy (poor) = 0.769
- Information Gain = 0.9905 - (0.225)(0) - (0.325)(0.8865) - (0.45)(0.769) = 0.6599

- Example 2: Term feature
- Entropy (3 years) = 0.72
- Entropy (5 years) = 0.88
- Information Gain = 0.9905 - (0.5)(0.72) - (0.5)(0.88) = 0.1905

- Decision: Split on Credit feature since it has the highest Information Gain (0.6599)

### Feature Split Selection Algorithm

- Given a subset of data M (a node in a tree)
- For each feature hi(x):
- Split data of M according to feature hi(x)
- Compute classification error OR Information gain split

- Choose feature h*(x) with lowest classification error or highest Information gain

### Recursion and Stopping Conditions

- Tree learning = Recursive stump learning
- Stopping conditions:
- All data agrees on y
- Already split on all features

### Prediction Algorithm

- Decision tree prediction algorithm
- Greedy decision tree learning

## Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.