Introduction to Machine Learning

Introduction to Machine Learning

• Machine learning (ML) involves computers modifying their actions to improve accuracy based on data.
• An online retail store uses client purchase and preference data to predict what users might be interested in.
• Prediction problems involve using existing data to forecast future actions.
• Supervised learning employs a teacher to guide the learning process.
• Storing large amounts of movement data is a known problem that is computationally challenging to extrapolate insights from.
• 2-D data and small datasets make data classification easy

Applications of Machine Learning

• Spam filtering
• Voice recognition
• Computer games
• Automatic number plate recognition
• Anti-skid braking systems
• Vehicle stability control
• Security applications

Learning Concepts

• Learning involves adapting, remembering, and generalizing from data.
• Generalization means recognizing similarities between different situations to apply knowledge across contexts.
• Intelligence incorporates reasoning and logical deduction.
• A key aspect of intelligence is learning and adapting.

Machine Learning Accuracy

• Accuracy in ML is measured by how well chosen actions reflect correct ones.
• Playing chess against a computer is an example of machine learning.
• Initially, a person beats the machine, then the machine learns and starts winning.
• Computational complexity is of interest and is broken into two parts: complexity of training, and complexity of applying a trained algorithm.

Types of Machine Learning

• Feature selection, which are the variables to use, is crucial for problem-solving in ML.
• Supervised Learning: Provides training data with correct responses (targets). The algorithm then generalizes to respond correctly to all possible inputs (also called exemplars).
• Unsupervised Learning: Does not provide correct responses, but the algorithm categorizes inputs by identifying similarities (density estimations).
• Reinforcement Learning: Is a mix between supervised and unsupervised learning. The algorithm finds out if the answer is right or wrong and aims to explore and get it right through trial and error. Also referred to as learning with a critic.
• Evolutionary Learning: Is based on biological evolution and is focused on fitness which relates to how good the current solution is.

Data Collection and Preparation

• Data is sometimes readily available, although most times, it must be collected.
• Having large amounts of data is a must in machine learning, however, it can prove challenging.
• Sensors can be subject to noise, making obtaining clean and concise data challenging.
• Enough data should be provided in order to ensure computation is feasible.

Machine Learning Process

• Feature Selection: Involves identifying useful traits and requires knowledge of the problem and data, without high noise or expensive collection.
• Algorithm Choice: Requires selecting the suitable algorithm for the dataset.
• Evaluation and model selection: Is the model selection process for experimentating with the correct values
• Training: Requires computational resources to build a model for output prediction.
• Evaluation: Requires that the built system is tested.

Machine Learning Terminology

• Inputs: Data from input vectors provided to the algorithm, where the input dimension.
• Weights: These are weighted connections between nodes. Weights in neural networks (a machine learning approach) are analogous to synapses in the brain and are arranged in a weight matrix W.
• Outputs: Output vectors, where j is the output dimension.
• Targets: Target vectors, extra data used in supervised training, that provide correct answers for the algorithm to learn

Activation Function

• For a neural network, the activation function is a mathematical threshold determining whether a neuron activates.

Error

• A function that computes the inaccuracies of the network of outputs in comparisons to the target data

Weight Space

• Plotting data is useful if <= 3 dimensions
• Plotting weights is useful in neural networks.
• Weights in neural networks are a set of corrdinates in weight space.
• Weight space can be used to assess how closely related neurons are related to the input.
• Plotted inputs can be changed in location of the neuron or used with nearby neurons being close to decide when it shoudl fire.

Curse of Dimensionality

• This curse applies to ML algorithms and the number of input dimension increases.
• With limited data, the algorithm will try to split it into out data.
• Data points are needed when additional features are added.

Data Sets

• Used to train and test an algorithm based on targeted supervised data.
• A reduction of data occurs because of the amount used for training and testing.

Overfitting

• Enough training makes algorithms generalize.
• Overtraining relates to the amounts of undertraining.
• Trained data is overfitted when noise and inaccuracies are learned.
• Stopping learning ensures prevention of over fitting.
• New data must be used to detect overfitting, specifically a validation data set.
• Cross validation is a statistical approach.

Data Percentages

• Some data sets require specific percentages of large or small amounts of data.
• Testing (large) - 25%
• Testing (small) - 20%
• Validation (large) - 25%
• Validation (small) - 20%
• Training (large) - 50%
• Training (small) - 60%

Confusion Matrix

• A square matrix containing all possible classes.
• Includes horizontal and vertical directions.
• Left hand-side = target metrics
• Top-side = predicted outputs

Measuring Matrix

• Data is placed into class by algorithm.
• Diagonal column is the correct metrics while the remaining are misclassifications.

Metrics

• Observe predictions using mathematical analytics.
• True positive = correct observation in class 1
• False positive = incorrect class observation
• True negative = correct class in class 2
• False negative = incorrect class in class 2
• Accuracy equates to true positives + true negatives / the total

Rates

• Sensitivity = true positive rate
• Number of correcct positive examples
• Incorrect identified false negatives in samples
• Specificity equals true negative rate.
• Precision equates to % or rate of correct true examples in positive examples.
• F1 Score summarizes performance metrics.

Receiver Operation Characteristic (ROC) Curve

• This graph plot highlights true positives to false positives.
• It also evaluates classifiers.
• An ideal classifier would highlight 100% true positives and no false positives
• While a poor classifier would highlight no true positives but mostly false positives.
• Working classifiers end up along a diagonal and require measurement of distance to line.

Accuracy

• Standard accuracy metrics are based on similar amounts of both positive & negative experiments.
• Values are based a balanced data set
• Matthew's Correlation Coefficient is a more accurate predictor metric.

Data Distribution

• Class properties should be fairly separable
• Overlapping datasets are difficult to differentiate between
• While separate ones are fairly distinct

Quantization

• English has much more data that can be used for analysis
• The more data relates to higher occurrence of events.

Calculating the Histogram and Class

• First, calculate a joint probability.
• Measure how often bin falls in histogram
• Or look at specific bins and measure class examples.
• Second, calculate Conditional Probability
• How likely a specific set of measurements align with what is expected.
• Or counting the number of items in the histogram and dividing by examples of class

Probability

• Bayes' Rule relates conditional probabililty to existing information.
• posterior probability helps to see what's important
• prior probablity relates to data in training set.
• Class is measured along features in trainingset.
• Loss matrix helps with calculating risk.

Randomness

• Is an experiment that gives an unpredictable output.
• Continuous is not finite.

Experiment

• Assigns experiment an experiment in space with probability.
• Requires a probabilistic distribution and is continuous.
• Can be found with functions defined with probability.
• Where variable are exactly equal to value.

Statistics

• Expected values are shown with averages that happen a lot of times.
• Weighted via averages and are common to values.
• Variance shows relationship in values.
• While square root variations show standard deviation.

Covariance

• Generalizing to see relationship via different variables.
• Is used to measure correlation and has matrix defined relationship.
• Elements and symmetry, with data relationship and dimensionality.

Example

• Check part via datasets and use location spread as a metric.

Mahalanobis distance

• Check that data is tightly controlled.
• And also that certain point in time is not important.
• With inverse covariance, vectors are set to euclidean distance.

Train Data

• Model helps to improve choices via parameters.
• More data = more freedom / complicated
• Data has bias depending on model
• Models accuracy should be precise due to lots of variation

Analysis

• Line has accuracy depending on bias
• While a spline can have increases and potential for less variance.