Artificial Intelligence with Machine Learning (Unit 1) PDF

Page 1 of Effective Date 01.06.2024 Page No....

Page 1 of Effective Date 01.06.2024 Page No. 17 VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) COURSE MATERIAL ARTIFICIAL INTELLIGENCE WITH MACHINE LEARNING Department COMPUTER SCIENCE Programme B.Sc Cyber Security ARTIFICIAL INTELLIGENCE WITH 24U3CYC04 Course Title MACHINE LEARNING Course Code Semester & Class & Section II B.SC (CYS) A III & 2024-25 Academic Year ASSISTANT Handling Staff M.DEVI Designation PROFESSOR Staff In charge HoD Principal  VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Page 2 of Effective Date 01.06.2024 Page No. 17 Unit - I What is Machine Learning? Machine Learning (ML) is a branch of artificial intelligence (AI) that focuses on the development of algorithms and statistical models that enable computers to perform tasks without explicit instructions. Instead, these algorithms and models learn from and make predictions or decisions based on data. Key Concepts in Machine Learning 1. Data: The raw information used for training machine learning models. Data can come in various forms, such as text, images, audio, or structured data like tables. 2. Model: A mathematical representation of a real-world process. Models are trained using data and can then be used to make predictions or decisions. 3. Training: The process of feeding data to a machine learning algorithm to help it learn patterns or relationships within the data. 4. Features: Individual measurable properties or characteristics of the data used by the model to make predictions. 5. Labels: The outcomes or target values that the model is trying to predict. 6. Algorithms: Procedures or sets of rules followed by the machine learning model to make decisions. Common algorithms include linear regression, decision trees, and neural networks. 7. Evaluation: Assessing the performance of a machine learning model using metrics such as accuracy, precision, recall, and F1-score. Types of Machine Learning 1. Supervised Learning: The model is trained on a labeled dataset, which means that each training example is paired with an output label. Examples include regression and classification tasks. 2. Unsupervised Learning: The model is trained on an unlabeled dataset, and it tries to learn the patterns and structure from the data without explicit labels. Examples include clustering and dimensionality reduction. 3. Semi-Supervised Learning: Combines a small amount of labeled data with a large amount of unlabeled data during training. This approach is useful when labeling data is expensive or time-consuming. 4. Reinforcement Learning: The model learns by interacting with an environment and receiving feedback in the form of rewards or penalties. It is commonly used in areas like robotics and game playing. Applications of Machine Learning 5. Image and Speech Recognition: Used in facial recognition systems, voice-activated assistants, and handwriting recognition. 6. Natural Language Processing (NLP): Powers applications like chatbots, language translation, and sentiment analysis. 7. Recommender Systems: Suggests products, movies, or content to users based on their preferences and past behavior.  Page 3 of Effective Date 01.06.2024 Page No. 17 VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) 8. Healthcare: Helps in diagnosing diseases, personalizing treatment plans, and predicting patient outcomes. 9. Finance: Used for credit scoring, fraud detection, and algorithmic trading. 10. Autonomous Vehicles: Enables self-driving cars to perceive and navigate the environment. WHY USE MACHINE LEARNING ? Machine Learning (ML) is employed for various compelling reasons across different domains. Here are some of the key benefits and motivations for using ML: 1. Handling Large and Complex Data Scalability: ML can analyze vast amounts of data quickly and efficiently, far beyond human capabilities. Complex Patterns: ML algorithms can detect intricate patterns and relationships within data that may not be evident through traditional analysis. 2. Automation of Routine Tasks Efficiency: Automating repetitive and mundane tasks can free up human resources for more strategic activities. Consistency: ML models perform tasks with consistent accuracy and without fatigue, reducing errors associated with manual processes. 3. Improved Decision-Making Data-Driven Insights: ML provides actionable insights based on data analysis, aiding in informed decision- making. Predictive Analytics: Models can forecast future trends and outcomes, helping businesses and individuals to plan better. 4. Personalization Customized Experiences: ML can tailor recommendations and experiences to individual users based on their preferences and behaviors (e.g., personalized content on streaming services, targeted advertising). Dynamic Adaptation: Systems can adapt in real-time to changing user preferences and behavior patterns. 5. Enhanced Accuracy Error Reduction: ML models, once trained, can reduce the likelihood of errors compared to manual processes, especially in tasks like data entry, anomaly detection, and quality control. Improved Diagnostics: In fields like healthcare, ML can assist in diagnosing diseases with higher accuracy by analyzing medical images and patient data. 6. Cost Efficiency Resource Optimization: Automating tasks reduces the need for extensive human labor, cutting operational costs. Efficiency Gains: Faster and more accurate processing of data can lead to significant time and cost savings.  VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Page 4 of Effective Date 01.06.2024 Page No. 17 7. Real-Time Applications Instant Responses: ML enables real-time decision-making and responses in applications such as fraud detection, stock trading, and autonomous driving. Dynamic Systems: Systems that adapt and respond instantly to new data and changing environments (e.g., dynamic pricing, real-time recommendation systems). 8. Innovative Applications New Capabilities: ML drives innovation by enabling new applications and services that were not previously possible (e.g., self-driving cars, intelligent virtual assistants, advanced robotics). Competitive Advantage: Businesses leveraging ML can gain a competitive edge through improved efficiency, innovation, and customer satisfaction. EXAMPLES OF ML APPLICATIONS: 1. Healthcare: Predicting patient outcomes, diagnosing diseases, and personalizing treatments. 2. Finance: Detecting fraudulent transactions, credit scoring, and algorithmic trading. 3. Retail: Inventory management, sales forecasting, and personalized marketing. 4. Manufacturing: Predictive maintenance, quality control, and process optimization. 5. Entertainment: Content recommendation, trend analysis, and audience engagement. 6. Transportation: Route optimization, autonomous driving, and traffic management. SUPERVISED LEARNING & UNSUPERVISED LEARNING By leveraging ML, organizations can harness the power of data to drive efficiencies, make more informed decisions, and innovate, thus improving overall performance and achieving strategic goals. SUPERVISED LEARNING Definition: Supervised learning involves training a model on a labeled dataset, which means that each training example is paired with an output label. The goal is for the model to learn a mapping from inputs to outputs so it can predict the labels for new, unseen data. Key Concepts: Training Data: Consists of input-output pairs, where the output (label) is known. Model Training: The model learns from the training data by adjusting its parameters to minimize the error in predicting the output labels. Evaluation: The model's performance is typically evaluated using metrics such as accuracy, precision, recall, and F1-score on a separate validation or test dataset. Common Algorithms:  Page 5 of Effective Date 01.06.2024 Page No. 17 VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Linear Regression: Predicts a continuous output variable based on input features. Logistic Regression: Used for binary classification tasks. Decision Trees and Random Forests: Tree-based models that can be used for both regression and classification. Support Vector Machines (SVM): Finds the optimal hyperplane for classification tasks. Neural Networks: Flexible models capable of learning complex patterns in data, used in tasks ranging from image recognition to language processing. Applications: Spam Detection: Classifying emails as spam or not spam. Sentiment Analysis: Determining the sentiment (positive, negative, neutral) of text data. Medical Diagnosis: Predicting diseases based on patient data. Stock Price Prediction: Forecasting stock prices based on historical data. Object Recognition: Identifying objects in images or videos. UNSUPERVISED LEARNING Definition: Unsupervised learning involves training a model on a dataset without labeled responses. The goal is to explore the structure of the data, often by clustering or reducing the dimensionality of the data. Key Concepts: Training Data: Consists of input data without labeled outputs. Model Training: The model tries to identify patterns, structures, or relationships in the data without supervision. Evaluation: The performance is often evaluated qualitatively or by using metrics specific to the task (e.g., silhouette score for clustering). Common Algorithms: K-Means Clustering: Partitions data into K distinct clusters based on similarity. Hierarchical Clustering: Builds a tree of clusters by recursively merging or splitting existing clusters. Principal Component Analysis (PCA): Reduces the dimensionality of the data by transforming it into a new set of orthogonal variables. t-Distributed Stochastic Neighbor Embedding (t-SNE): Reduces dimensions while preserving the local structure of the data, often used for visualization. Autoencoders: Neural networks used to learn compressed representations of data. Applications: Customer Segmentation: Grouping customers based on purchasing behavior. Anomaly Detection: Identifying unusual patterns or outliers in data. Market Basket Analysis: Finding associations between items in large datasets. Document Clustering: Organizing documents into groups based on content similarity. Image Compression: Reducing the size of image files while retaining important features.  VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Page 6 of Effective Date 01.06.2024 Page No. 17 Comparison Aspect Supervised Learning Unsupervised Learning Labeled Data Requires labeled data (input-output pairs) Does not require labeled data Find hidden patterns or intrinsic structures in Goal Predict outputs for new data data Linear/Logistic Regression, Decision Trees, K-Means, Hierarchical Clustering, PCA, t-SNE, Algorithms SVM, Neural Networks Autoencoders Spam detection, sentiment analysis, medical Customer segmentation, anomaly detection, Applications diagnosis market basket analysis Evaluation Accuracy, precision, recall, F1-score Silhouette score, inertia, explained variance Metrics Both supervised and unsupervised learning have their unique strengths and are used based on the specific requirements and nature of the problem being addressed. Batch learning and online learning are two paradigms of machine learning that describe how models are trained on data. BATCH LEARNING Definition: Batch learning, also known as offline learning, involves training a machine learning model on the entire dataset at once. This means that all the data is available and used for training the model in one or more passes. Key Characteristics: Full Dataset: The model is trained using the entire dataset in one go or in large chunks (batches). Computational Resources: Typically requires substantial computational resources because it processes the whole dataset. Fixed Model: Once trained, the model is fixed and does not update until it is retrained with new data. Training Time: The training process can be time-consuming, especially with large datasets. Advantages: Stability: Since the model is trained on the entire dataset, it can achieve a high level of accuracy and stability.  Page 7 of Effective Date 01.06.2024 Page No. 17 VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Optimization: Allows for the use of sophisticated optimization algorithms that require access to the whole dataset. Disadvantages: Resource Intensive: Requires significant memory and computational power. Inflexibility: Not suitable for situations where data arrives continuously or the model needs to be updated frequently. Applications: Image Recognition: Training deep learning models on large datasets of images. Natural Language Processing (NLP): Building language models or text classifiers using extensive text corpora. Predictive Analytics: Creating predictive models using historical data for long-term forecasting. ONLINE LEARNING Definition: Online learning, also known as incremental learning, involves training a machine learning model incrementally as new data arrives. The model updates its parameters continuously with each new data point or small batch of data. Key Characteristics: Incremental Updates: The model is updated with each new data point or mini-batch of data. Low Latency: Capable of making quick updates, making it suitable for real-time applications. Memory Efficient: Requires less memory as it processes one data point at a time or small batches. Advantages: Adaptability: The model can adapt to new data quickly, making it suitable for dynamic environments where data is continuously generated. Efficiency: Requires less memory and computational power since it processes small amounts of data at a time. Disadvantages: Stability: The model may be less stable as it constantly updates with each new data point, potentially leading to higher variance. Complexity: Implementing online learning algorithms can be more complex, and careful tuning is required to balance learning rate and stability. Applications: Stock Trading: Continuously updating models to predict stock prices based on the latest market data. Recommendation Systems: Adapting recommendations in real-time based on user behavior and interactions. Fraud Detection: Identifying fraudulent activities as new transaction data comes in.  VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Page 8 of Effective Date 01.06.2024 Page No. 17 Robotics: Allowing robots to learn from their interactions with the environment in real-time. Comparison Aspect Batch Learning Online Learning Data Processing Processes the entire dataset at once Processes data incrementally Computational Load High, requires substantial resources Lower, processes small chunks of data Model Updates Fixed model until retrained Continuous updates with new data Memory Usage High, as it needs to load the entire dataset Low, processes one data point or mini-batch at a time Latency Higher, as it needs to process all data Low, suitable for real-time applications Stability High, due to comprehensive training data Variable, depends on new incoming data Both batch learning and online learning have their own advantages and are suited to different types of problems and operational environments. The choice between them depends on the specific requirements of the application, such as the availability of data, computational resources, and the need for real-time updates. Instance-based and model-based learning are two fundamental approaches to supervised learning in machine learning. They differ in how they generalize from training data to make predictions on new data. INSTANCE-BASED AND MODEL-BASED LEARNING Instance-Based Learning Definition: Instance-based learning, also known as memory-based learning or lazy learning, involves storing the training data and using it directly to make predictions without building an explicit model. When a prediction is required, the algorithm compares the new instance with the stored instances to find the most similar ones. Key Characteristics: No Explicit Model: The training phase is minimal; most of the work is done during the prediction phase. Similarity Measure: Predictions are made based on the similarity (or distance) between the new instance and stored instances. Lazy Learning: Since it defers computation until prediction, it’s called lazy learning.  Page 9 of Effective Date 01.06.2024 Page No. 17 VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Common Algorithms: K-Nearest Neighbors (KNN): Predicts the label of a new instance based on the majority label of its k-nearest neighbors. Locally Weighted Learning: Makes predictions based on a weighted average of the nearby instances. Advantages: Simplicity: Easy to implement and understand. Adaptability: Can quickly adapt to new data without retraining. Flexibility: Can model complex relationships without an explicit function. Disadvantages: Scalability: Can be computationally expensive and slow for large datasets, as it requires comparing the new instance with many stored instances. Storage: Requires storing the entire training dataset. Noise Sensitivity: Sensitive to noisy data, as it directly uses the training instances. Applications: Recommendation Systems: Recommending products based on the preferences of similar users. Medical Diagnosis: Classifying diseases based on the symptoms of similar past cases. Pattern Recognition: Recognizing patterns like handwritten digits or spoken words. MODEL-BASED LEARNING Definition: Model-based learning involves building an explicit model that generalizes from the training data to make predictions on new data. This model captures the underlying relationships and patterns in the data through a learning process. Key Characteristics: Explicit Model: An explicit mathematical or computational model is created during the training phase. Parameter Estimation: The model has parameters that are learned from the training data. Eager Learning: The computational effort is concentrated in the training phase, while prediction is typically fast. Common Algorithms: Linear Regression: Models the relationship between input features and output using a linear equation. Logistic Regression: Models the probability of a binary outcome. Decision Trees: Builds a tree-like model of decisions based on input features. Neural Networks: Learns complex patterns using layers of interconnected nodes.  VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Page 10 of Effective Date 01.06.2024 Page No. 17 Advantages: Efficiency: Predictions are generally faster once the model is trained. Scalability: Can handle large datasets more efficiently after the initial training. Generalization: Typically better at generalizing from the training data to new instances, especially with well- regularized models. Disadvantages: Training Time: Can require significant time and computational resources to train. Complexity: Model selection, parameter tuning, and training can be complex processes. Overfitting: Risk of overfitting if the model is too complex for the given amount of data. Applications: Economic Forecasting: Predicting economic indicators like inflation or GDP growth. Image Classification: Recognizing objects or scenes in images. Natural Language Processing (NLP): Tasks like sentiment analysis, machine translation, and text generation. Fraud Detection: Identifying fraudulent transactions based on patterns learned from historical data. Comparison Aspect Instance-Based Learning Model-Based Learning Approach Uses training data directly Builds an explicit model from training data Computation Most effort during prediction Most effort during training Storage Stores entire training dataset Stores a model with learned parameters Scalability Can be slow with large datasets More efficient with large datasets after training Adaptability Quickly adapts to new data Requires retraining or updating the model Noise Sensitive to noisy data Can be regularized to reduce sensitivity to noise Sensitivity K-Nearest Neighbors, Locally Weighted Linear Regression, Decision Trees, Neural Examples Learning Networks Instance-based and model-based learning serve different purposes and are chosen based on the specific requirements and constraints of the problem at hand. Instance-based methods are often preferred for their simplicity and adaptability, while model-based methods are chosen for their efficiency and ability to generalize. MAIN CHALLENGES OF MACHINE LEARNING:  Page 11 of Effective Date 01.06.2024 Page No. 17 VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Machine learning (ML) presents several challenges that can impact the effectiveness and efficiency of developing and deploying ML models. Here are some of the main challenges: 1. Data Quality and Quantity Challenge: ML models require large amounts of high-quality data to learn effectively. Poor quality data (e.g., containing errors, missing values, or noise) can lead to inaccurate models. Solutions: Data cleaning and preprocessing to handle missing values and correct errors. Data augmentation techniques to artificially increase the size of the dataset. Collecting more data to improve the model's performance. 2. Overfitting and Underfitting Challenge: Over fitting: The model performs well on training data but poorly on new, unseen data. Under fitting: The model is too simple to capture the underlying patterns in the data. Solutions: Use cross-validation techniques to evaluate model performance. Apply regularization methods (e.g., L1, L2) to penalize complex models. Ensure the model complexity matches the data complexity. 3. Scalability Challenge: Deploying ML models to handle large-scale data in production environments can be challenging. Solutions: Use scalable architectures and technologies (e.g., Hadoop, Spark). Implement model optimization techniques to reduce latency. Continuously monitor and update the model to maintain performance. 4. Security and Privacy Challenge: Protecting sensitive data and ensuring privacy in ML applications, especially in areas like healthcare and finance. Solutions: Implement data anonymization and encryption techniques. Use privacy-preserving ML methods like differential privacy and federated learning. Follow ethical guidelines and regulations for data usage.  VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Page 12 of Effective Date 01.06.2024 Page No. 17 5. Model Selection Challenge: Choosing the appropriate algorithm for a given problem can be difficult, as different algorithms have different strengths and weaknesses. Solutions: Perform a thorough comparison of various models using performance metrics. Use ensemble methods to combine multiple models. Leverage automated machine learning (AutoML) tools to assist in model selection. UNIT – I COMPLETED UNIT – II SUPERVISED LEARNING Classification and Regression Regression and Classification algorithms are Supervised Learning algorithms. Both the algorithms are used for prediction in Machine learning and work with the labeled datasets. But the difference between both is how they are used for different machine learning problems. The main difference between Regression and Classification algorithms that Regression algorithms are used to predict the continuous values such as price, salary, age, etc. and Classification algorithms are used to predict/Classify the discrete values such as Male or Female, True or False, Spam or Not Spam, etc.  Page 13 of Effective Date 01.06.2024 Page No. 17 VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Classification: Classification is a process of finding a function which helps in dividing the dataset into classes based on different parameters. In Classification, a computer program is trained on the training dataset and based on that training, it categorizes the data into different classes. Example: The best example to understand the Classification problem is Email Spam Detection. The model is trained on the basis of millions of emails on different parameters, and whenever it receives a new email, it identifies whether the email is spam or not. If the email is spam, then it is moved to the Spam folder. Types of ML Classification Algorithms: Classification Algorithms can be further divided into the following types: o Logistic Regression o K-Nearest Neighbours o Support Vector Machines o Kernel SVM o Naïve Bayes o Decision Tree Classification o Random Forest Classification  VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Page 14 of Effective Date 01.06.2024 Page No. 17 Regression: Regression is a process of finding the correlations between dependent and independent variables. It helps in predicting the continuous variables such as prediction of Market Trends, prediction of House prices, etc. The task of the Regression algorithm is to find the mapping function to map the input variable(x) to the continuous output variable(y). Example: Suppose we want to do weather forecasting, so for this, we will use the Regression algorithm. In weather prediction, the model is trained on the past data, and once the training is completed, it can easily predict the weather for future days. Types of Regression Algorithm: o Simple Linear Regression o Multiple Linear Regression o Polynomial Regression o Support Vector Regression o Decision Tree Regression o Random Forest Regression Difference between Regression and Classification Regression Algorithm Classification Algorithm In Regression, the output variable must be of In Classification, the output variable must be a discrete continuous nature or real value. value. The task of the regression algorithm is to map the The task of the classification algorithm is to map the input input value (x) with the continuous output value(x) with the discrete output variable(y). variable(y). Regression Algorithms are used with continuous Classification Algorithms are used with discrete data. data.  Page 15 of Effective Date 01.06.2024 Page No. 17 VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) OVERFITTING AND UNDERFITTING IN MACHINE LEARNING Overfitting and Underfitting are the two main problems that occur in machine learning and degrade the performance of the machine learning models. The main goal of each machine learning model is to generalize well. Here generalization defines the ability of an ML model to provide a suitable output by adapting the given set of unknown input. It means after providing training on the dataset, it can produce reliable and accurate output. Hence, the underfitting and overfitting are the two terms that need to be checked for the performance of the model and whether the model is generalizing well or not. Before understanding the overfitting and underfitting, let's understand some basic term Signal: It refers to the true underlying pattern of the data that helps the machine learning model to learn from the data. Noise: Noise is unnecessary and irrelevant data that reduces the performance of the model. Bias: Bias is a prediction error that is introduced in the model due to oversimplifying the machine learning algorithms. Or it is the difference between the predicted values and the actual values. Variance: If the machine learning model performs well with the training dataset, but does not perform well with the test dataset, then variance occurs. What is Machine Learning Generalization? Generalization in ML refers to the model's ability to adapt properly to new, previously unseen data, drawn from the same distribution as the one used to create the model. It is the concept that a machine learning model should not only fit the training data well, but also perform well on unseen data. Generalization is crucial because it determines how well the learned knowledge can be applied to future scenarios, which is the ultimate goal of machine learning.  VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) Page 16 of Effective Date 01.06.2024 Page No. 17 OVERFITTING Overfitting occurs when our machine learning model tries to cover all the data points or more than the required data points present in the given dataset. Because of this, the model starts caching noise and inaccurate values present in the dataset, and all these factors reduce the efficiency and accuracy of the model. The overfitted model has low bias and high variance. The chances of occurrence of overfitting increase as much we provide training to our model. It means the more we train our model, the more chances of occurring the overfitted model. Overfitting is the main problem that occurs in supervised learning. Example: The concept of the overfitting can be understood by the below graph of the linear regression output: As we can see from the above graph, the model tries to cover all the data points present in the scatter plot. It may look efficient, but in reality, it is not so. Because the goal of the regression model to find the best fit line, but here we have not got any best fit, so, it will generate the prediction errors. HOW TO AVOID THE OVERFITTING IN MODEL Both overfitting and underfitting cause the degraded performance of the machine learning model. But the main cause is overfitting, so there are some ways by which we can reduce the occurrence of overfitting in our model. o Cross-Validation o Training with more data o Removing features o Early stopping the training o Regularization o Ensembling  Page 17 of Effective Date 01.06.2024 Page No. 17 VIVEKANANDHA COLLEGE OF ARTS AND SCIENCES FOR WOMEN (Autonomous) UNDERFITTING Underfitting occurs when our machine learning model is not able to capture the underlying trend of the data. To avoid the overfitting in the model, the fed of training data can be stopped at an early stage, due to which the model may not learn enough from the training data. As a result, it may fail to find the best fit of the dominant trend in the data. In the case of underfitting, the model is not able to learn enough from the training data, and hence it reduces the accuracy and produces unreliable predictions. An underfitted model has high bias and low variance. Example: We can understand the underfitting using below output of the linear regression model: As we can see from the above diagram, the model is unable to capture the data points present in the plot. HOW TO AVOID UNDERFITTING: o By increasing the training time of the model. o By increasing the number of features.