Data Science Fundamentals Quiz

Data Science

• Data science applies computational and statistical methods to gain insights into real-world problems.
• The core components include raw data and a well-defined problem statement.
• Model choice is determined by maximizing performance on a specific evaluation metric.
• The macro-steps are data collection, data analysis, and data presentation.
• Data analysis includes exploration, model selection, and testing.
• Data presentation aims to maximize information while minimizing unnecessary detail.
• Data management provides essential support, including efficient storage, retrieval, and suitable infrastructure.

Statistics

• A correlation coefficient measures the linear relationship between two variables, ranging from -1 to 1.
• A value of 1 indicates a positive correlation, 0 indicates no correlation, and -1 indicates a negative correlation.
• Pearson’s correlation coefficient, corr(X, Y), is the ratio of covariance to the product of standard deviations.
• The Bonferroni principle warns against discovering meaningless patterns in data analysis.
• A sample is a subset of a population, and statistical analysis aims to draw conclusions about the entire population from the sample.
• Selection bias occurs when a non-representative sample is chosen for analysis.
• Simpson's paradox shows trends observable in data groups that disappear when the groups are combined, indicating confounding variables.
• Descriptive statistics summarizes observed phenomena, useful for quick data analysis (e.g., mean).
• Inferential statistics generalizes results from data to the entire population through hypothesis testing and model building.
• A frequentist approach formulates a hypothesis, estimates a confidence level, repeats experiments, and computes test statistics.
• A Bayesian approach applies probability to statistical problems, finding the probability of a hypothesis using Bayes’ theorem.
• Hypothesis testing evaluates the likelihood of a hypothesis being true, given the data.
• The common approach to hypothesis testing involves computing the probability of observing the data given the null hypothesis (p-value).
• The null hypothesis is rejected if the p-value is less than a threshold (T).
• The central limit theorem states that the sample mean of any random variable approaches a normal distribution as sample size increases.
• Bayes' theorem updates the probability of an event given new information, expressed as p(A|B) = p(A) p(B|A)/p(B).
• Lindley’s paradox indicates that Bayesian and frequentist approaches can yield different results based on prior distribution choices.

Data Visualization

• Anscombe’s quartet highlights the limitations of descriptive statistics by showing datasets with identical stats but different graphs.
• The Gestalt principles demonstrate that the whole is different from the sum of its parts.
• Closure: Perceiving separate elements as a complete figure.
• Common fate: Grouping objects moving in the same direction as a unit.
• Continuity: Perceiving disconnected figures as a continuous form.
• Proximity: Perceiving objects near each other as a group.
• Similarity: Grouping similar elements into collective entities.
• Symmetry: Perceiving symmetry in objects regardless of distance.
• A histogram plots counts or frequencies of a variable.
• A bar plot compares the same metric across different categories, where bars can be reordered.
• A pie chart represents parts/proportions of a whole.
• A scatter plot analyzes the relationship between two continuous variables, revealing trends or correlations.
• A box plot visualizes the distribution of values, showing the mean and symmetry of the dataset.
• A heatmap represents matrix entries with colors and is useful for correlations/distances between entities.
• A bubble chart is a scatter plot variation that can represent more than two dimensions using color and point diameter.

ML Introduction

• The three main machine learning paradigms are supervised, unsupervised, and reinforcement learning.
• Supervised learning finds a mapping between input and output variables using labeled data.
• Classification involves a discrete output, while regression involves a continuous output.
• Unsupervised learning uses unlabeled data to cluster, find patterns, and identify important features.
• The confusion matrix measures prediction model performance, reporting true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN).
• Accuracy is the ratio of correct predictions (TP+TN) to all predictions (TP+TN+FP+FN).
• Precision is the ratio of true positives (TP) to all positive predictions (TP+FP).
• Recall is the ratio of true positives (TP) to all actual positives (TP+FN).
• F1 score is calculated as 2PR/(P+R).
• A recall-precision curve evaluates classification model performance, particularly for imbalanced datasets.
• A receiver operating characteristic (ROC) curve assesses binary classification model performance, particularly for balanced datasets.
• Regression metrics include Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and Mean Absolute Deviation (MAD).
• Model selection chooses the best hyperparameters, splitting data into training, validation, and test sets.
• K-fold cross-validation splits data into k folds, using one as a test set and the rest for training and validation over k turns.
• Micro-averaging sums TPs, FPs, and FNs before computing metrics.
• Macro-averaging computes per-fold metrics and then averages them.
• Overfitting occurs when a model learns the training set too well but fails to generalize to new examples.
• Data pre-processing tasks include aggregation, cleaning, discretization, normalization, conversion, and integration.

ML KNN

• KNN classifies a variable's target based on the most frequent target value of its k nearest neighbors, using a distance function for similarity.
• Distance metrics include Euclidean and Hamming distance.
• KNN can be used for regression when the variable is continuous by averaging the target values of the k nearest neighbors.
• KNN requires numeric features.
• KNN has high computational complexity in high-dimensional spaces.

ML Decision Trees

• Decision Trees are widely used ML models that classify data based on attribute-value representations.
• Each internal node represents an attribute.
• Each branch corresponds to a value of that attribute.
• Each leaf node represents a class.
• Can be viewed as a set of classification rules, with each path from the root to a leaf representing one such rule.
• The C4.5 algorithm builds decision trees by choosing attributes, partitioning the dataset, applying C4.5 recursively, and stopping.
• C4.5 handles continuous attributes and uses information gain ratio.
• Metrics that aid feature selection include the Gini index and information gain ratio.
• The Gini index measures the impurity of a node, with a best value of 0.
• Information gain measures the expected reduction in entropy, with higher values indicating better splits.
• Information gain ratio normalizes information gain, solving the bias issue of information gain and the Gini Index.
• Pruning reduces overfitting by removing unnecessary branches using pre-pruning (stopping tree growth early) and post-pruning (removing subtrees).
• Decision trees are interpretable and handle mixed data types but are prone to overfitting.
• Random forests are ensembles of decision trees trained on random feature subsets.
• Ensemble models combine multiple weak classifiers, including bagging, boosting, and stacking.

Linear classifiers

• Linear classifiers separate data using a linear decision boundary.
• They are useful for large datasets where simplicity and low computational cost are crucial.
• The perceptron is a simple neural network model with a single neuron, capable of classifying linearly separable examples.
• A loss function measures the difference between model predictions and actual target values.
• Gradient descent minimizes the loss function by iteratively adjusting model parameters.
• The learning rate determines the step size in parameter updates, requiring careful tuning to avoid slow convergence or instability.
• The perceptron can be used for regression by removing the threshold activation function and using a different loss function.
• Logistic regression estimates the probability that an input belongs to a class using the sigmoid function.
• The output of logistic regression is the probability, and feature weights indicate the importance and direction of the feature's influence.

Support Vector Machines

• SVMs find an optimal separating hyperplane to divide data points into classes.
• SVMs aim to maximize the margin, which is the distance between the hyperplane and the closest data points.
• A soft-margin SVM allows some misclassification using slack variables.
• The C hyper-parameter balances margin size and classification errors.
• Large C values try to classify all points correctly, which can lead to overfitting; small C values allow misclassifications, improving generalization.
• Regularization prevents overfitting by adding a penalty term to the loss function.
• Kernel machines extend SVMs to non-linear problems by transforming the input space.
• Kernel machines use the "kernel trick" to compute dot products in a higher-dimensional space efficiently.
• Polynomial and Gaussian (RBF) kernels are commonly used.
• Multi-class classification can be handled with one-vs-all (OvA) or one-vs-one (OvO) approaches.

Neural Networks

• Neural Networks (NNs) are used for function approximation.
• NNs minimize a loss function, such as binary or multinomial cross-entropy.
• A feed-forward neural network (FNN) consists of an input layer, hidden layers, and an output layer.
• Each neuron applies a function: f(x;θ)=g(w⋅x+b), where g is a non-linear activation function.
• Activation functions introduce non-linearity, enabling the network to learn complex relationships.
• Common activation functions include Sigmoid, Softmax, and ReLU.
• Neural networks are trained using supervised learning with labeled data.
• Backpropagation computes gradients for all parameters, enabling efficient training through gradient descent.
• Early stopping prevents overfitting by monitoring validation error.
• Convolutional Neural Networks (CNNs) are used for image classification, employing convolutional layers to learn spatial hierarchies.
• Recurrent Neural Networks (RNNs) are used for sequential data processing, such as time series forecasting and NLP.

Regression

• Regression predicts real-valued continuous variables, unlike classification, which predicts discrete categories.
• Parametric regression requires assuming a function shape in advance.
• Linear regression is an example of Parametric regression.
• Non-parametric regression doesn't assume a function shape but requires more data.
• K-Nearest Neighbors (KNN) regression is an example of Non-parametric regression.
• K-Nearest Neighbors, linear regression, decision trees, neural networks and support vector machines can be adjusted for regression.

Unsupervised learning

• Clustering groups data points based on similarity, maximizing intra-cluster and minimizing inter-cluster similarity.
• The K-means algorithm assigns data points to the closest centroid.
• The K-means algorithm computes new centroids based on the mean of assigned points.
• K-means is dependent on random initialization.
• K-means requires choosing the number of clusters (K) in advance.
• K-means assumes linear separation.
• Clustering performance measures include the Dunn Index, Silhouette Score, Homogeneity, Completeness, and V-Measure.

Association Rule Mining

• Data mining is the process of transforming data into intelligible knowledge.
• The confidence of a rule is the probability of consequent happening, given the antecedent.
• An association rule implies X -> Ik, where X is a subset of items and Ik is an item not in X.
• The support of an itemset is the fraction of transactions containing the itemset.
• The support of a rule is the fraction of transactions containing both the antecedent and consequent.
• The lift of a rule measures the support of the rule relative to the expected support if the items were independent.
• Rule with Lift > 1 indicates positive dependence.
• Interest measures the difference between the confidence of the rule and the support of the consequent.
• Apriori identifies frequent items, generates candidate itemsets, prunes infrequent itemsets, and generates rules.
• A dendrogram is a tree-like diagram that visualizes the hierarchical structure of clusters.
• Single Linkage calculates the minimum distance between any two points from each cluster, able to capture elongated structures.
• Principal Component Analysis (PCA) reduces dimensionality by projecting data into a lower space, preserving main characteristics.

Reinforcement Learning (RL)

• RL involves an agent learning to achieve goals by interacting with an environment and receiving rewards for its actions.
• Markov Decision Process (MDP)
• MDP models sequential decision-making with states, actions, rewards, and transition functions.
• The learning goal in single-agent RL maximizes the expected return, often discounted to prioritize immediate rewards
• Discount factor determines the significance of future rewards—a value near 0 prioritizes immediate rewards, and one near 1 values future rewards almost equally
• State value function measures the expected value of being in a state.
• State-action value function measures the expected value of taking an action in a state.
• A greedy policy selects the action with the highest expected value.
• Dynamic programming computes optimal policies, assuming full knowledge of the MDP.
• Temporal difference learning learns optimal policies without knowing the transition or reward functions beforehand.
• Learning rate determines how much the model updates its estimates.
• The common assumption made to prove convergence of RL algorithms is infinite exploration.

Multi-Agent Reinforcement Learning (MARL)

• MARL introduces challenges over RL through Non-stationarity, credit assignment, and equilibrium selection.
• A zero-sum game is where the sum of agent's rewards is always zero.
• A joint policy specifies the actions of all agents in every state.
• A best response policy maximizes its expected return given fixed policies of other agents.
• Nash equilibrium means no agent can improve its expected return by unilaterally changing its policy.
• Independent learning is simple, but suffers from non-stationarity issues.
• Central learning avoids non-stationarity, with increased computational complexity.
• CTDE shares information during training but acts independently.
• Deep learning in RL enables generalization and efficient representation of complex environments.
• In RL, the training target changes continuously (a moving target problem) unlike supervised learning.
• RL data isn't i.i.d., which can cause overfitting to recent experiences.