Untitled

Overview of Machine Learning & Its Applications and Data Preprocessing

• Artificial intelligence and machine learning history will be discussed
• Definition and types of machine learning will be examined: Supervised, Unsupervised and Reinforcement Learning
• Applications of machine learning, like Natural Language Processing (NLP) and Computer Vision (CV) will be presented
• Data Preprocessing will be covered

Artificial Intelligence

• The term "artificial intelligence" was first coined in 1956 at the Dartmouth Conference
• Homo sapiens can control other species due to their thinking ability
• Programs exhibiting behaviors considered intelligent when shown by humans are called intelligent - according to Herbert Simon
• AI involves solving exponentially hard problems in polynomial time by leveraging domain knowledge - according to Elaine Rich

Human Intelligence vs Artificial Intelligence

• Humans can learn and apply knowledge and skills, while AI recreates human intelligence in machines

Classical Problems in AI

• Turing Test is a test of a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a humans.
• Alan Turing questioned if machines can think; named it "too meaningless"
• Turing's "Imitation Game" is now known as the Turing Test (1912–1954)
• The Loebner Prize is an annual competition where chatbots responding like humans are judged; The grand price of USD 100,000 is still open.
• A question explored in "The Chinese Room Argument" is Can an agent locked in a room, processing Chinese questions based on syntactic rules, be said to understand Chinese?

Topics in AI

• AI topics include: natural language understanding, speech recognition/synthesis, image processing, qualitative reasoning, computer vision, knowledge representation, neural networks, machine learning, knowledge discovery, graphics, logic, search, models, planning, problem solving, tactile sensing, adversarial reasoning and robot control

Machine Learning

• Machine Learning is a subset of AI
• ML enables computers to learn from data and past experiences to make predictions

Machine Learning History

• 1950s: Arthur Samuel created the Checker's program
• 1960s: Neural Network and Rosenblatt's Perceptron came about; limitations of Perceptron were proven by Minsky et al.
• 1970s: AI Winter began, which was also known as AAAI (Association for the Advancement of Artificial Intelligence)
• 1980s: Backpropagation came to be via David Rumelhart et al.
• 1990s: Support Vector Machines were developed
• 2010s: Deep Learning was introduced
• 2020s: Generative AI developed

Dataset

• Datasets include Labels, Independent and Dependent Variables, such as age(years), weight(kg), systolic BP(mm Hg) of subjects

Types of Learning Algorithms

• Deductive Learning involves hard-coded rules being provided. An example includes multiplying numbers; you can apply this to any two numbers and find the answer
• Inductive Learning involves examples being provided for you, with needs to learn the mapping from input to output. Supervised learning is an example.

Types of Machine Learning Algorithms

• Supervised Learning Classification answers the question "is this A or B?"
• Supervised Learning Regression answers the question "how much or how many?"
• Unsupervised Learning Clustering answers the question "how is this organized?"
• Reinforcement Learning answers the question "what should I do next.

Examples of Regression

• Supervised Learning Algorithms can predict a continuous variable's value
• Examples include employee salary as a function of experience, and price of a bike as a function of age
• A Machine Learning model example includes y = f(x) also can be written as Salary = f(experience) or f = w₀ + w₁x where f is dependent output, x is independent input variable(s), w₀ is y-axis intercept, and w₁ is coefficient

Examples of Classification

• Predict a binary (discrete) variable's value using Supervised Learning Algorithms
• Examples include if a given image is outdoor or indoor, if a personal loan application can be approved or not, and if a given cancer cell is begin or malignant.

Examples of Clustering

• The Un-supervised Learning Algorithm is used when there is no variable to predict; only the common pattern to group. Examples include grouping different customers based on income and spending.

Examples of Association Rule Learning

• Un-Supervised Learning Algorithms such as Recommender Systems are used to recognize customers with similar shopping patterns, recommend products having high probability of buying

Examples of Reinforcement Learning

• Learning is through reward and penalty when using this Algorithm

Spectrum of Supervision

• The Spectrum of supervision increases with "semi" supervised and fully supervised data

Applications of Machine Learning: NLP

• Natural Language Processing (NLP) is a branch of computer science/AI that allows computers to understand text and spoken words. Computer science, linguistics, and machine learning are combined to study how computers and humans communicate.
• Sentiment analysis determines if data is positive, negative or neutral
• Text summarization condenses text to a shorter version while keeping the content meaning. News headlines and result snippets in web searches are some uses.
• Machine translation automatically translates text from one language to another

Sentiment Analysis

• Sentiment analysis determines whether data is positive, negative, or neutral in business to understand customer needs and monitor brand/product sentiment

Language Translation

• Automatically translate from one language to another, using NLP techniques to understand the structure and meaning of the original text and creating a translation conveying the meaning in the target language

Applications of Machine Learning: Computer Vision

• Computer vision enables computers to understand and interprets the visual world. It utilizes digital images and deep learning models to identify and classify objects, and then react to what it "sees":
• Facial Recognition: Identifying individuals through visual analysis
• Self-driving cars use computer vision to navigate and avoid obstacles
• Robotic Automation enables decisions based on visual input so that the robot can perform tasks.
• Person detection is useful for intelligent perimeter monitoring
• Medical Imaging uses computer vision for diagnostic applications, cancer treatments, surgery and research

Applications of Machine Learning: Computer Vision

• Includes object recognition and object detection

Object Recognition

• Object Recognition maps any given sample to one of the categories, such as indoor/kitchen and outdoor/beach.

Data Preprocessing

• Data is a set of observations or measurements used to train and test a ML model.
• Data can be any unprocessed value, text, speech, image, or video
• Types of data: normal, ordinal, discrete and continuous data

Data and Types of Data

• Nominal Data (Categorical): Represents categories without inherent order/ranking, ex: Gender & Colors.
• Ordinal Data: Represents categories with a meaningful order/ranking, ex: Education level (High School < Bachelor's < Master's < Ph.D.)
• Discrete Data: Numerical data that can take on specific, distinct values, ex: number of cars in a parking lot.
• Continuous Data: Numerical data that can take any value within a given range, often involving decimals, ex: Weight, Temperature and Time.

Why is Data Preprocessing Required?

• Raw data quality is poor
• Real-world datasets are noisy, missing and/or inconsistent

Handling Missing Data

• Missing data, also known as missing values, is where some of the observation in a data set are blank
• Delete the missing values by deleting the entire row or column
• Impute the missing values by replacing with an arbitrary value like 0, or a central tendency measure for that column

Scaling and Data Normalization

• Data scaling is a preprocessing technique used to transform the values of features with differing scales in a dataset to a similar scale
• The goal of data scaling is so that all features contribute equally to the mode, reducing outlier impact, and enabling a faster model convergence
• Techniques include normalization and standardization

Importance of Feature Scaling

• Number of bedrooms (1 to 10) and house size (500 to 5000) must be scaled for their values to contribute equally to the model when predicting house prices

Normalization

• A goal is to transform features to be on a similar scale
• Min-max normalization: transform feature values from their natural range (example, 100 to 900) into a standard range, which is typically [0, 1] (or sometimes [-1,1])
• To rescale the feature values between arbitrary values [a,b]:

Xscaled = a + (X - Xmin) (b-a) /  Xmax - Xmin

where min-max normalization is suggested when the feature is uniformly distributed across a fixed range

Standardization

• A Feature is standardized if its values have zero-mean and unit-variance
• This is known as Z-Score normalization method where the values are centered around a zero mean with a unit standard deviation.
• Where x1 = x - μ / σ, μ is represents the mean of the feature values and o represents the standard deviation

Normalization vs. Standardization

• Normalization rescales value to a range between 0 and 1 and is sensitive to outlier
• Standardization centers data around a zero mean, scales it to a standard deviation of 1 and has less sensitivity to outliers

Dealing with Outliers

• This is another data preprocessing step in which outliers need to be detected, then handled. Training and accuracy of a ML model can decrease because of them.

Detecting and Handling Outliers

• Ways to detect outliers is through visualization/box-plot, using standard deviation or Z-scores
• Ways to handle outliers: remove the, use quantile-based capping and flooring, or Mean/Median imputation

Encoding Categorical Values

• Encoding methods for Ordinal and Nominal values are required for ML Models
• Some of the encoding methods can be used: Label, One-Hot, Effect, Binary, Base N, and Target encoding

Encoding Categorical Variables: Label Encoding

• Each label is converted into an integer value.
• If we apply label encoding to 'safety' feature, ['low', 'med', 'high'] will be encoded to [1, 2, 0].
• Similarly for 'lug_boot' feature, ['small', 'med', 'big'] will be encoded to [2, 1, 0]

Encoding Categorical Variables: One-Hot Encoding

• Each category value is mapped with a binary value of either 0 or 1.

Terminology

• A feature vector is a numerical representation of an object
• Class to which X belongs to is y ∈ Y, and needs to be estimated, based on the training set
• Task is to design a classifer or decision rule of f: X → Y which decides about the class label based on X.

Features and Samples: An Example

• Feature Vector: A vector of observations
• Class to which X belongs: overweight, normal
• Training Set: To need to estimate, based on training set
• Task: to design a classifer (decision rule) in order to classify the person, classify him/her. f: X → Y
• Sample: Each X is considered to be as a sample.

Training, Validation, and Test sets

• Training set: examples are used to fit/learn the parameters of a model; can also be used to build a perceptron classifier
• Validation set: It is a development set to used to tune the hyperparameters and architecture of a model.
• Test set: This set is a learned machine learning model and its performance is assessed.

Feature Engineering

• Raw data is transformed into features to create a predictive model using machine learning.
• Feature Transformation transforms the features into a more suitable representation for the machine learning model.
• Feature Selection selects a subset of relevant features used for machine-learning model training, where some info is removed
• Example: For a dataset having a model to classify person overweight or normal; if this is the case, then the name is removed