Artificial Intelligence and Computational Systems

• Artificial Intelligence (IA) seeks to develop systems capable of performing tasks that require human intelligence
• These tasks include pattern recognition, decision making, and learning

History and Evolution of AI

• 1950s: AI was born with the first chess-playing and math-solving programs
• 1960s-1970s: Increased use of rule-based systems and search algorithms
• 1980s-1990s: Expert systems and neural networks emerged
• 21st Century: Deep Learning advancements led to AI being applied across various industries

Key Concepts in Computational Systems and Modeling

• Computational System: A combination of hardware, software, and peripherals working together to execute specific tasks and processes
• Model: A complex object that takes an input parameter and generates an output, and can represent real systems through equations or data structures
• Equation: A mathematical representation of a function, sometimes used to define math models
• Algorithm: A set of steps or instructions inputted into a model for calculations or data processing

Types of Artificial Intelligence

• Weak AI (Narrow AI): Designed for specific tasks like virtual assistants or facial recognition
• Strong AI (General AI): Possesses cognitive abilities similar to humans
• Superintelligent AI: Hypothetical intelligence superior to human intelligence
• Reactive AI: Systems without memory that respond to real-time stimuli, like AI used in chess games
• Limited Memory AI: Can use past experiences to make decisions, such as self-driving cars with recent environment data analysis
• Theory of Mind AI: Under development, aims to understand and respond to human emotions, simulating empathy
• Self-Aware AI: Hypothetical AI with self-awareness and the ability to understand its existence and impact

Main AI Techniques

• Machine Learning: Algorithms that learn from data without being explicitly programmed
• Neural Networks: Models inspired by the human brain for pattern recognition
• Natural Language Processing (NLP): Techniques that enable machines to understand and generate human language
• Computer Vision: Analysis and interpretation of images and videos

Intelligence in Machines

• Intelligence in machines is their capacity to perform tasks normally requiring human cognitive skills.
• Problem Solving: AI systems can analyze complex data sets, identify patterns, and make logical deductions
• Learning: AI algorithms can learn from data and experiences, improving performance with automatic and deep learning over time
• Perception: Technologies like computer vision and natural language processing allow machines to perceive and understand the world through visual and linguistic inputs
• Decision Making: AI systems make decisions based on input data, predefined rules, or learned patterns, optimizing for outcomes in various areas
• Automation: AI facilitates task automation across fields, streamlining processes and enhancing efficiency
• Adaptability: Some AI systems adjust to changes in their environment or input data, modifying their behavior or responses accordingly

AI Applications

• Medicine: AI-assisted diagnostics
• Finance: Fraud detection
• Automobiles: Autonomous driving
• Industry: Process automation

AI Ethics and Challenges

• Privacy and security: Protecting personal data
• Job displacement: Impact on human employment
• Decision making: Responsibility in critical decisions
• Bias in algorithms: Possible discrimination in AI models

What is Data Science?

• Data Science is an interdisciplinary field combining statistics, programming, and domain knowledge to extract knowledge or insights from data
• The goal of Data Science is to transform raw data into useful information to inform decision-making

Data Science seeks to:

• Predict future phenomena
• Reduce costs
• Mitigate risks
• Increase profits
• Improve process efficiency and decision-making

Origin of data:

• Physical or environmental sensors
• Online transactions (e.g., web purchases)
• Social networks (comments, likes, posts)
• Mobile devices (GPS, apps, activity)
• Institutional databases (hospitals, banks, governments)

Data processing:

• Data proceeds from collection, cleaning, exploratory analysis, modeling (e.g. machine learning), and results visualization

Where is Data Science used?

• Data Science is used in multiple industries including health, finance, marketing, and technology
• Data scientists use tools such as Python, R, SQL, and frameworks like Pandas, Scikit-learn, and TensorFlow

General Protocol in Data Science

• The data science process follows a structured flow to ensure data-driven decisions are reliable and useful
• The typical protocol includes the following

Steps in the Protocol

• Problem Identification: Clearly define the question to answer or the problem to solve
• Data Collection: Obtain data from various sources such as sensors, databases, APIs, and surveys
• Data Cleaning: Remove duplicates, handle missing values, correct errors, and transform formats
• Exploratory Data Analysis (EDA): Visually and statistically explore data to understand structure, detect patterns, correlations, outliers, etc.
• Feature Selection/Engineering: Choose or create the most relevant variables (features) to help the model to make good predictions
• Machine Learning Application: Select and train machine learning models on processed data
• Model Evaluation and Adjustment: Evaluate the model's performance with appropriate metrics and adjust hyperparameters or test other models if needed
• Communication and storytelling: The findings are presented clearly through visualizations, reports, or dashboards
• The goal is for non-technical stakeholders to understand and act based on the insights

Machine Learning

• Machine Learning is a sub-discipline of AI that allows machines to learn from data and improve their performance without explicit programming

Machine learning models can be classified into:

• Supervised Learning: The model is trained with labeled data
• Unsupervised Learning: The model seeks unlabeled data patterns

Supervised Learning

• Supervised learning aims to teach a machine to make predictions or decisions based on historical data with correct answers
• If we want a computer to recognize emails as 'spam' or 'not spam', we provide many examples, and the model attempts to identify and generalize these patterns for new emails without labels

Workflow in supervised learning:

• Data Gathering: Collect data relevant to the problem
• Preprocessing: Cleaning the data, handling missing values, transforming categorical variables into numerical ones, and scaling the values
• Data Division: Data is split into training (70-80%) and testing or validation (20-30%) sets
• Model Selection: Choose the proper algorithm according to the kind of data and problem
• Model Training: the model analyzes the data and fine-tunes internal parameters for useful patterns
• Model Evaluation: test how can the model predict using test data through metrics measures.
• Prediction: Finally, the model is used to make predictions on new data without labels

Unsupervised Machine Learning

• In unsupervised learning, the model seeks underlying data structure, like groupings or hidden relationships
• Clustering (grouping data in similar groups), dimensionality reduction (reducing number of characteristics), feature selection and association analysis are the key factors
• Common models include: K-means clustering, PCA, and Autoencoders

Training, Validation, and Test Sets in Machine Learning

• In machine learning, the data is divided into different sets to ensure the model generalizes well and does not overfit the training data

These sets are

• Training Set: Data used to train, and for the model adjusts its parameters
• Validation Set: Data used to evaluate model performance during training and adjust hyperparameters.
• Test Set: Testing of the final model and its final performance

How to split?

• The data set is generally divided between 70-80% for the training set, 10-15% for the validation set, and 10-15% for the test set

Classification vs Regression

• Classification involves predicting a category or class
• Regression involves predicting a real number

Classification Models

• K-Nearest Neighbors: used for classifying a new data point based on the "k" nearest data points
• Naive Bayes: Utilizes probability theory (Bayes' theorem) to estimate the most probable class of a data point
• Logistic Regression: used for binary classification (two classes)

Regression Models

• Linear Regression - finds the best fitting line to the data to predict an output

Evaluation of Models

• Classification
• Confusion Matrix: shows model's rights/wrongs for each class
• Precision: out of all the times the model said "yes", how many times it was correct
• Recalll: of all the real positive values, how many were correctly identified
• F1-score: Balance between precision and recall
• Accuracy: overall percentage of correct models
• Regression
• MAE: Average of the absolute error, easy to analyze
• MSE: Squares the error, thus penalizing larger errors
• RMSE: The root sqaure of MSE, with the same units as the original variable
• R^2 Score: Perfect R^2 score is 1.

Overfitting vs Underfitting

• Overfitting: when the model learns the training data so well that it cannot generalize new data

Model Complexity

• Complex Model: Has more parameters for the model to train over, often leading to overfitting
• Simple Model: has small parameters, possible leading to underfitting.

Bias-variance Tradeoff

• Balance the tradeoff between bias and variance.
• Balance should lead to minimized errors. Resulting in an efficient and accurate outcome

Curse of Dimensionality

• As the number of data characteristics increase, the data exponentially increases. Often leading to overfitting due to the model learning complex patterns
• Dimensionality reduction and feature selection helps combat dimensionality

Sample Size

• Overfitting is more prone when low samples are taken
• A larger sample size provides more to analyze and minimizes overfitting.

Cross-Validation

• Used to evaluate models and the better one through less data skewing.

K-Fold Cross Validation

• Divides the set into k equal parts (folds) to average the results for a better measure.

Stratified K-Fold Cross-Validation

• The same as K-Fold but maintains the ratio of classes in each fold to maintain balance.
• Used in problems that are hard to classify

Leave-One-Out Cross-Validation

• Very precise yet has a high cost.

Imbalanced Datasets

• One set in the data is more prevalent than, skewing the data
• Achieves a high accuracy, but is useless in the minor set

Downsampling

• Set the majority dataset to be smaller

Upweighting

• Weigh certain errors to provide results

Oversampling

• Generate samples of the smaller dataset or minority

Threshold Adjustment

• Modify the decision threshold to provide more accurate results

Confusion Matrix

• Used to measure the rate of accuracy of a model through True Positives, True Negatives, False Positives and False Negatives

Key Metrics

• Precision: (TP / (TP + FP)); Of the times you predicted "positive", how many times were you correct?
• Recall: (TP / (TP + FN)); Of the true positives, how many did you get?
• F1 Score : Harmonic Average of Precision and Recall (2 x (Precision x Recall) / (Precision + Recall))
• Accuracy : (TP + TN) / Total; can be misleading if the classes are imbalanced
• False Positive Rate : FP / (FP + TN); Ratio of the real negatives where classified incorrectly

Popular tools used in Python for Machine Learning

• Scikit-learn: contains algorithms, evaluation training methods and preprocessing
• Pandas: used to transform the data in tables for easier reading
• Numpy: allows for vector and array math
• Matplotlib: creates data visualization through graphs