Decision Trees and Ensemble Learning Quiz

Decision Trees, Bagged and Boosted Decision Trees in Supervised Learning

• Bootstrapping involves sampling with replacement, where some data is left out of each tree in the sample.
• A bag of decision trees using subspace sampling is known as a random forest.
• Boosting aggregates weak learners to form a strong predictor by adding new trees that minimize the error of previous learners.
• Decision trees predict responses by following decisions in the tree from the root to a leaf node, using branching conditions and trained weights.
• Bagged decision trees consist of independently trained trees on bootstrapped data, while boosting adds weak learners iteratively.
• Decision trees are formed by rules based on variables in the data set, with splitting stopping when no further gain can be made or pre-set stopping rules are met.
• CART produces classification or regression trees, depending on the dependent variable's nature.
• Classification trees represent class labels on leaves and conjunctions of features on branches, while regression trees predict continuous values.
• A loss function is minimized to fit a decision tree, choosing the best variable and splitting value among all possibilities.
• Criteria like maximum depth, node size, and pruning are used to ensure interpretability and prevent overfitting in decision trees.
• Technical indicators like volatility, short-term and long-term momentum, short-term reversal, and autocorrelation regime are used as independent variables in a financial market context.
• Random forests are ensembles of random trees, like bootstrapping with decision trees using randomly selected features.

Decision Trees, Bagged and Boosted Decision Trees, and Technical Indicators in Market Analysis

• Bootstrapping involves sampling with replacement, leaving some data out in each tree, and is used to create a random forest with subspace sampling.
• Random forests are a collection of decision trees using subspace sampling, while boosting aggregates weak learners to form a strong predictor over time.
• A boosted model adds new trees to minimize errors by previous learners, fitting new trees on residuals of previous trees.
• Decision trees predict data responses by following branching conditions and trained weights, and can be pruned for model simplification.
• Bagged decision trees and boosting combine weaker trees into stronger ensembles, with bagging using independent trees and boosting iteratively adding weak learners.
• Decision trees are formed by rules based on variables' values, recursively splitting nodes until no further gain or stopping rules are met.
• Classification and regression trees (CART) are non-parametric techniques producing either classification or regression trees based on the dependent variable.
• Classification trees represent class labels in leaves and conjunctions of features in branches, while regression trees predict continuous values.
• Decision trees are built by minimizing a loss function, considering the best variable and splitting value and using criteria to ensure interpretability and prevent overfitting.
• Technical indicators in market analysis include volatility, short-term momentum, long-term momentum, short-term reversal, and autocorrelation regime.
• Each technical indicator has binary outcomes, and their combinations can be used to predict market behavior and probabilities of returns.
• Random forests are created by bootstrapping with decision trees and randomly selecting features, while bootstrapping involves sampling with replacement to create subsets of data.