Chapter-II-Data-Collection-and-Management.pdf

IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Republic of the Philippines Isabela State University...

IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Republic of the Philippines Isabela State University Echague, Isabela College of Computing Studies, Information and Communication Technology Chapter II - Data Collection and Management Data collection and management is the process of gathering and organizing data in a way that it can be used to answer questions, make decisions, or solve problems. It is an essential part of many research projects, business operations, and government initiatives. The data collection process typically involves the following steps: 1. Define the research question or problem. What do you want to learn from the data? 2. Identify the data sources. Where will you get the data? 3. Choose the data collection methods. How will you collect the data? 4. Collect the data. This may involve conducting surveys, interviews, or experiments. 5. Clean and prepare the data. This involves removing errors and inconsistencies from the data. 6. Store and manage the data. This involves organizing the data in a way that it can be easily accessed and analyzed. 7. Analyze the data. This involves using statistical methods to draw insights from the data. 8. Communicate the results. This involves sharing the findings of the data analysis with others. Data management is the ongoing process of organizing, storing, and protecting data. It is important to ensure that the data is accessible to authorized users, and that it is protected from unauthorized access, use, or disclosure. Data collection and management is a complex and challenging process, but it is essential for organizations to make informed decisions and achieve their goals. Here are some of the benefits of data collection and management: Improved decision-making: Data can help organizations make better decisions by providing insights into their operations, customers, and markets. Increased efficiency: Data can help organizations identify areas where they can improve efficiency, such as by reducing costs or increasing productivity. Enhanced innovation: Data can help organizations identify new opportunities for innovation by providing insights into customer needs and trends. Improved compliance: Data can help organizations comply with regulations by providing a record of their activities. Enhanced security: Data can help organizations protect themselves from security threats by providing a way to track and monitor access to sensitive data. 1 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. There are many different methods of data collection, each with its own advantages and disadvantages. Some of the most common methods include: Surveys: Surveys are a popular way to collect data from a large number of people. They can be conducted online, by mail, or in person. Interviews: Interviews are a more in-depth way to collect data from a smaller number of people. They can be conducted face-to-face, over the phone, or by video chat. Experiments: Experiments are used to test cause-and-effect relationships. They involve manipulating one variable and observing the effects on another variable. Observational studies: Observational studies are used to collect data without interfering with the subjects being studied. They can be conducted in a natural setting or in a laboratory. The best data collection method for a particular project will depend on the research question, the budget, and the time constraints. Data management is an ongoing process that involves organizing, storing, and protecting data. There are many different data management tools and techniques available, and the best choice will depend on the specific needs of the organization. Data collection and management is a critical part of many research projects, business operations, and government initiatives. By following the steps outlined above, organizations can ensure that they are collecting and managing data in a way that is efficient, effective, and compliant. Data Source A data source is the location where data originates from. It can be internal or external to an organization. Internal data sources include: Transactional data: This is data about the day-to-day operations of an organization, such as sales, orders, and inventory. Customer data: This is data about customers, such as demographics, purchase history, and contact information. Employee data: This is data about employees, such as compensation, performance reviews, and training history. Financial data: This is data about the financial performance of an organization, such as revenue, expenses, and profits. External data sources include: Government data: This is data collected by governments, such as census data and economic data. Industry data: This is data collected by industry associations, such as market research data and pricing data. 2 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Academic data: This is data collected by universities and research institutions, such as scientific data and medical data. Social media data: This is data collected from social media platforms, such as user posts, comments, and likes. The choice of data source will depend on the specific needs of the organization. For example, if an organization is trying to understand its customer behavior, it might use customer data from its CRM system. If an organization is trying to forecast demand, it might use economic data from a government agency. When choosing a data source, it is important to consider the following factors: Relevance: The data should be relevant to the research question or problem that the organization is trying to solve. Accuracy: The data should be accurate and reliable. Timeliness: The data should be up-to-date. Accessibility: The data should be easy to access and use. Cost: The cost of obtaining the data should be reasonable. Data sources can be classified into two main categories: primary and secondary. Primary data is data that is collected for the first time. It is typically collected through surveys, interviews, experiments, or observations. Secondary data is data that has already been collected and is available for reuse. It can be found in books, journals, government reports, and online databases. Primary data is often more accurate and reliable than secondary data, but it can be more expensive and time-consuming to collect. Secondary data is less expensive and time- consuming to collect, but it may not be as accurate or reliable as primary data. The best data source for a particular project will depend on the research question, the budget, and the time constraints. Data Collection and APIs APIs, or application programming interfaces, are a way for software applications to communicate with each other. They provide a set of rules and instructions that allow applications to request and receive data from each other. APIs can be used to collect data from a variety of sources, including: Web services: Web services are websites that provide data through APIs. For example, the Google Maps API allows you to get information about geographic locations, such as their latitude and longitude. Databases: APIs can be used to access data from databases. For example, the MySQL API allows you to query MySQL databases. 3 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Sensor devices: APIs can be used to collect data from sensor devices, such as temperature sensors or GPS sensors. Social media platforms: APIs can be used to collect data from social media platforms, such as Twitter or Facebook. To collect data using an API, you will need to: Find the API that you want to use. There are many APIs available, so you will need to do some research to find one that meets your needs. Get an API key. Most APIs require you to get an API key before you can use them. This key is used to authenticate your requests and to prevent unauthorized access to the data. Understand the API documentation. The API documentation will tell you how to use the API to request and receive data. Make API requests. Once you understand the API documentation, you can start making API requests to collect data. Data collection using APIs can be a quick and easy way to get the data that you need. However, it is important to be aware of the limitations of APIs. For example, some APIs may only allow you to access a limited amount of data, or they may charge a fee for access. Here are some of the benefits of using APIs to collect data: Speed: APIs can be used to collect data quickly and easily. Efficiency: APIs can automate the data collection process, which can save time and resources. Flexibility: APIs can be used to collect data from a variety of sources. Scalability: APIs can be scaled to meet the needs of large data sets. Here are some of the challenges of using APIs to collect data: Cost: Some APIs may charge a fee for access. Security: APIs can be a security risk if they are not properly secured. Complexity: APIs can be complex to use, especially if you are not familiar with them. Overall, APIs can be a valuable tool for data collection. However, it is important to weigh the benefits and challenges before deciding whether or not to use them. Exploring and Fixing Data Data exploration is the process of understanding the data by summarizing its main characteristics, identifying patterns, and outliers. It is an important first step in data analysis, as it helps to ensure that the data is clean and ready for further analysis. There are many different methods of data exploration, but some of the most common include: 4 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Data profiling: This involves summarizing the main characteristics of the data, such as the number of records, the number of variables, the data types, and the distribution of the values. Data visualization: This involves using charts and graphs to visualize the data, which can help to identify patterns and outliers. Statistical analysis: This involves using statistical tests to identify relationships between variables. Data fixing is the process of identifying and correcting errors in the data. It is an important step in data preparation, as it ensures that the data is accurate and reliable. There are many different methods of data fixing, but some of the most common include: Data cleaning: This involves removing errors from the data, such as typos, missing values, and inconsistent values. Data imputation: This involves filling in missing values with estimates. Data transformation: This involves converting the data into a different format, such as converting categorical data into numerical data. Data exploration and data fixing are essential steps in data analysis. By understanding the data and fixing any errors, you can ensure that your analysis is accurate and reliable. Here are some of the things to look for when exploring and fixing data: Missing values: Are there any missing values in the data? If so, how many? And are they missing randomly or systematically? Outliers: Are there any outliers in the data? Outliers are data points that are significantly different from the rest of the data. They can be caused by errors or by legitimate variation in the data. Duplicate values: Are there any duplicate values in the data? Duplicate values can occur when data is entered incorrectly or when two different records refer to the same entity. Inconsistent values: Are there any inconsistent values in the data? Inconsistent values are data points that have different values for the same variable. They can occur when data is entered incorrectly or when two different records refer to the same entity. Incorrect data types: Are there any data points that are stored in the wrong data type? For example, a date value might be stored as a string. Corrupt data: Is there any corrupt data in the file? Corrupt data is data that is damaged or unreadable. Once you have identified any problems with the data, you can take steps to fix them. For example, you can remove missing values, impute missing values, or transform data types. Data exploration and data fixing are important steps in data analysis. By taking the time to explore and fix the data, you can ensure that your analysis is accurate and reliable. 5 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Data storage management: What is it and why is it important? Effective data storage management is more important than ever, as security and regulatory compliance have become even more challenging and complex over time. Enterprise data volumes continue to grow exponentially. So how can organizations effectively store it all? That's where data storage management comes in. Effective management is key to ensuring organizations use storage resources effectively, and that they store data securely in compliance with company policies and government regulations. IT administrators and managers must understand what procedures and tools encompass data storage management to develop their own strategy. Organizations must keep in mind how storage management has changed in recent years. The COVID-19 pandemic increased remote work, the use of cloud services and cybersecurity concerns such as ransomware. Even before the pandemic, all those elements saw major surges -- and after the pandemic, these elements will still be prominent. With this guide, explore what data storage management is, who needs it, advantages and challenges, key storage management software features, security and compliance concerns, implementation tips, and vendors and products. What data storage management is, who needs it and how to implement it Storage management ensures data is available to users when they need it. Data storage management is typically part of the storage administrator's job. Organizations without a dedicated storage administrator might use an IT generalist for storage management. The data retention policy is a key element of storage management and a good starting point for implementation. This policy defines the data an organization retains for operational or compliance needs. It describes why the organization must keep the data, the retention period and the process of disposal. It helps an organization determine how it can search and access data. The retention policy is especially important now as data volumes continually increase, and it can help cut storage space and costs. The task of data storage management also includes resource provisioning and configuration, unstructured and structured data, and evaluating how needs might change over time. To help with implementation, a management tool that meets organizational needs can ease the administrative burden that comes with large amounts of data. Features to look 6 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. for in a management tool include storage capacity planning, performance monitoring, compression and deduplication. Advantages and challenges of data storage management Data storage management has both advantages and challenges. On the plus side, it improves performance and protects against data loss. With effective management, storage systems perform well across geographic areas, time and users. It also ensures that data is safe from outside threats, human error and system failures. Proper backup and disaster recovery are pieces of this data protection strategy. An effective management strategy provides users with the right amount of storage capacity. Organizations can scale storage space up and down as needed. The storage strategy accommodates for constantly changing needs and applications. Storage management also makes it easier on admins by centralizing administration so they can oversee a variety of storage systems. These benefits lead to reduced costs as well, as admins are able to better utilize storage resources. Benefits of data storage management include more efficient operations and optimized resource utilization. Challenges of data storage management include persistent cyberthreats, data management regulations and a distributed workforce. These challenges illustrate why it's so important to implement a comprehensive plan: A storage management strategy should ensure organizations protect their data against data breaches, ransomware and other malware attacks; lack of compliance could lead to hefty fines; and remote workers must know they'll have access to files and applications just as they would if in a traditional office environment. 7 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Distributed and complex systems present a hurdle for data storage management. Not only are workers spread out, but systems run both on premises and in the cloud. An on- premises storage environment could include HDDs, SSDs and tapes. Organizations often use multiple clouds. New technologies, such as AI, can benefit organizations but also increase complexity. Unstructured data -- which includes documents, emails, photos, videos and metadata -- has surged, and this also complicates storage management. Unstructured data challenges include volume, new types and how to gain value. Although some organizations might not want to spend the time to manage unstructured data, in the end it saves money and storage space. Vendors such as Aparavi, Dell EMC, Pure Storage and Spectra Logic offer tools for this type of management. Object storage can provide high performance but also has challenges, including the infrastructure's scale-out nature and potentially high latency, for example. Organizations must address issues with metadata performance and cluster management. Data storage management strategies Storage management processes and practices vary, depending on the technology, platform and type. Here are some general methods and services for data storage management: storage resource management software consolidation of systems multiprotocol storage arrays storage tiers strategic SSD deployment hybrid cloud scale-out systems archive storage of infrequently accessed data elimination of inactive virtual machines deduplication disaster recovery as a service object storage Organizations may consider incorporating standards-based storage management interfaces as part of their management strategy. The Storage Management Initiative Specification and the Intelligent Platform Management Interface are two veteran models, while Redfish and Swordfish have emerged as newer options. Interfaces offer management, monitoring and simplification. As far as media type, it's tempting to go all-flash because of its performance. However, to save money, try a hybrid drive option that incorporates high-capacity HDD and high- speed SSD technology. 8 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Organizations also must choose among object, block and file storage. Block storage is the default type for HDDs and SSDs, and it provides strong performance. File storage places files in folders and offers simplicity. Object storage efficiently organizes unstructured data at a comparatively low cost. NAS is another worthwhile option for storing unstructured data because of its organizational capabilities and speed. Understand how object, block and file storage compare. Storage security With threats both internal and external, storage security is as important as ever to a management strategy. Storage security ensures protection and availability by enabling data accessibility for authorized users and protecting against unauthorized access. A storage security strategy should have tiers. Security risks are so varied, from ransomware to insider threats, that organizations must protect their data storage in a number of ways. Proper permissions, monitoring and encryption are key to cyberthreat defense. Offline storage -- for example, in tape backup -- that isn't connected to a network is a strong way to keep data safe. If attackers can't reach the data, they can't harm it. While it's not feasible to keep all data offline, this type of storage is an important aspect of a strong storage security strategy. 9 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Another aspect is off-site storage, one form of which is cloud storage. Organizations shouldn't assume that this keeps their data entirely safe. Users are responsible for their data, and cloud storage is still online and thus open to some risk. The surge in remote workers produced a new level of storage security complications, including the following risks: less secure home office environments; use of personal devices for work; misuse of services and applications; less formal work habits; adjustments to working from home; and more opportunities for malicious insiders. Endpoint security, encryption, access controls and user training help protect against these new storage security issues. Data storage compliance Compliance with regulations has always been important, but the need has increased in the last few years with laws such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act. These laws specifically address data and storage, so it's incumbent on organizations to comprehend them and ensure compliance. 10 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. GDPR can spur enterprises into adopting practices that deliver long-term competitive advantages. Data storage management helps organizations understand where they have data, which is a major piece of compliance. Compliance best practices include documentation, automation, anonymization and use of governance tools. Immutable data storage also helps achieve compliance. Immutability ensures retained data -- for example, legal holds -- doesn't change. Vendors such as AWS, Dell EMC and Wasabi provide immutable storage. However, organizations should still retain more than one copy of this data, as immutability doesn't protect against physical threats, such as natural disasters. Data storage technology, vendors and products Key features for overall data storage management providers include resource provisioning, process automation, load balancing, capacity planning and management, predictive analytics, performance monitoring, replication, compression, deduplication, snapshotting and cloning. Recent trends among vendors include services for cloud storage and the container management platform Kubernetes. Top storage providers can support a range of different platforms. And though Kubernetes is more specialized, it has gained traction: Vendors such as Diamanti, NetApp and Pure Storage provide Kubernetes services. 11 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Some form of cloud management is essentially table stakes for storage vendors. A few vendors, including Cohesity and Rubrik, have made cloud data management a hallmark of their platforms. Many organizations use more than one cloud, so multi-cloud data management is crucial. Managing data storage across multiple clouds is complex, but vendors such as Ctera, Dell EMC, NetApp and Nutanix can help. Cloud management components include automation and orchestration; security; governance and compliance; performance monitoring; and cost management. The future of data storage management Data storage administrators must be ready for a consistently evolving field. Cloud storage was trending up before the pandemic and has skyrocketed since -- and once organizations go to the cloud, they typically stay there. As a result, admins must understand the various forms of cloud storage management, including multi-cloud, hybrid cloud, cloud-native data and cloud data protection. 12 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Hyper-convergence, composable infrastructure and computational storage are also popular frameworks. In addition, admins must be aware of other new and emerging technologies that help storage management, from automation to machine learning. Lesson Overview 1. Importance of Data Collection and Preparation 2. Basic Data Quality Assessment 3. Ethical Considerations in Data Collection 4. Importing Data from Various Sources 5. Data Cleaning and Preprocessing in Excel 6. Handling Missing Data and Outliers Lecture Outline Each topic is covered with a theoretical lecture followed by practical laboratory activities. The activities focus on applying the concepts in Microsoft Excel, allowing students to work with datasets related to their fields (Mathematics or Social Science). 1. Importance of Data Collection and Preparation Theoretical Lecture Objective: Understand why data collection and preparation are crucial in data science. Key Points: o Foundation of Data Science: Data collection is the first step in data analysis; the quality of collected data determines the reliability of results. o Minimizing Errors: Proper preparation helps identify and correct errors early in the analysis process. o Efficiency: Well-organized data saves time during the analysis stage. o Consistency: Consistent data allows for accurate comparisons across different datasets or studies. Laboratory Activities Task: Create a Data Collection Plan using Excel. Steps: 1. Define Data Requirements: Identify the type of data needed for analysis based on the student's major (e.g., test scores for Mathematics or survey responses for Social Science). 13 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. 2. Design a Data Collection Template in Excel: Set up a spreadsheet with columns for data sources, collection methods, data types, and validation rules. 3. Document the Collection Process: Write a step-by-step plan for collecting the data, including tools and procedures to be used. 2. Basic Data Quality Assessment Theoretical Lecture Objective: Learn to assess data quality using dimensions such as completeness, accuracy, and consistency. Key Points: o Completeness: Ensures all necessary data is present. o Accuracy: Data must be correct and free from errors. o Consistency: Data should follow the same format and standards. o Validity: Data should represent what it is supposed to measure. o Uniqueness: Avoids duplicate records that can skew analysis results. Laboratory Activities Task: Perform a Data Quality Assessment using Excel. Steps: 1. Import Data into Excel: Students import a dataset they have collected. 2. Check for Completeness: ▪ Use COUNTA and IF formulas to identify missing data. 3. Validate Data Accuracy: ▪ Use DATA VALIDATION to set rules for data entry (e.g., numerical ranges, text length). 4. Ensure Consistency: ▪ Use FIND and REPLACE to standardize text entries (e.g., "Male" and "Female" vs. "M" and "F"). 5. Check for Duplicates: ▪ Use the Remove Duplicates feature to identify and remove duplicates. 3. Ethical Considerations in Data Collection Theoretical Lecture Objective: Understand ethical implications when collecting and handling data. Key Points: o Informed Consent: Participants must be informed about the data collection process. o Data Privacy and Security: Protecting sensitive information is critical. 14 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. o Anonymity and Confidentiality: Ensure that participants' identities remain confidential. o Avoiding Bias: Collect data in a manner that avoids bias and misrepresentation. Laboratory Activities Task: Develop an Ethical Data Collection Plan. Steps: 1. Define the Purpose of the Data Collection: Document the objectives and intended use of the data. 2. Create a Consent Form Template in Excel: Include sections for study information, participant rights, and signature fields. 3. Design Data Storage and Access Control Measures: Plan how to secure data in Excel (e.g., using password protection and hidden sheets). 4. Importing Data from Various Sources Theoretical Lecture Objective: Learn how to import data from different formats into Excel. Key Points: o Data Formats: Understanding common formats such as CSV, JSON, and Excel workbooks. o Data Import Techniques: Using Excel's Get Data feature for importing data from various sources. o Combining Data: Methods for combining multiple datasets into one using Excel tools like Power Query. Laboratory Activities Task: Import Data from Various Sources. Steps: 1. Import from CSV Files: ▪ Go to Data > Get Data > From Text/CSV and select a file. 2. Import from Web Pages: ▪ Use Get Data > From Web to import data directly from a URL. 3. Combine Multiple Datasets: ▪ Use Power Query to merge or append datasets from different files. 5. Data Cleaning and Preprocessing in Excel Theoretical Lecture Objective: Learn essential data cleaning and preprocessing techniques. Key Points: 15 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. o Removing Duplicates: Ensures unique data points. o Handling Missing Data: Using techniques like mean imputation or data interpolation. o Standardizing Data Formats: Ensuring consistency in data types (e.g., date formats). o Data Validation: Setting up rules to prevent incorrect data entry. Laboratory Activities Task: Clean and Preprocess Data in Excel. Steps: 1. Remove Duplicates: ▪ Select the data range and go to Data > Remove Duplicates. 2. Handle Missing Data: ▪ Use IF and ISBLANK functions to fill in missing values. 3. Standardize Data Formats: ▪ Use TEXT functions and Data Validation to ensure consistent formatting. 4. Create Data Validation Rules: ▪ Go to Data > Data Validation to set up entry restrictions (e.g., only allowing numbers between 0 and 100). 6. Handling Missing Data and Outliers Theoretical Lecture Objective: Learn methods to handle missing data and outliers. Key Points: o Handling Missing Data: Options include deletion, mean/mode imputation, and interpolation. o Identifying Outliers: Use visual tools like box plots and scatter plots. o Dealing with Outliers: Deciding whether to remove, cap, or transform outliers based on their impact on the analysis. Laboratory Activities Task: Handle Missing Data and Outliers in Excel. Steps: 1. Identify Missing Data: ▪ Use CONDITIONAL FORMATTING to highlight missing values. 2. Impute Missing Data: ▪ Apply the AVERAGEIF function to replace missing values. 3. Identify Outliers: ▪ Use Excel’s BOX PLOT feature to visualize and detect outliers. 4. Transform or Remove Outliers: 16 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. ▪ Use IF statements to cap outliers or remove them based on a defined threshold. Summary and Assessment 1. Review Key Concepts: Summarize the importance of data collection, quality assessment, ethical considerations, importing data, cleaning, and handling missing data and outliers. 2. Quiz: Conduct a short quiz using Excel to test understanding. 3. Assignment: Each student collects a dataset, performs a quality assessment, cleans the data, handles missing data and outliers, and submits a report detailing the steps and insights. Lab Activity 1: Importing and Assessing Data Quality Dataset: Sales Data (CSV) Description: This dataset includes sales information with some inconsistencies and missing values. It helps students practice importing data and performing basic data quality assessments. Example Data: OrderID Date Product Quantity Price 1001 15/01/2024 Widget A 10 20.5 1002 16/01/2024 Widget B 25 1003 17/01/2024 Widget C 5 15 1004 18/01/2024 10 1005 19/01/2024 Widget A 8 21 1006 20/01/2024 Widget B 12 24 1007 Widget C 7 16 Lab Activity Instructions: Import the CSV file into Excel. Assess the quality of the data by calculating summary statistics (mean, median, mode) for numerical columns. Use conditional formatting to highlight missing values and inconsistencies. 17 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Lab Activity 2: Data Cleaning and Preprocessing Dataset: Customer Feedback (Excel) Description: This dataset includes customer feedback with various issues such as extra spaces, inconsistencies in text, and varying formats. It is used for data cleaning and preprocessing. Example Data: CustomerID Date Feedback Rating 1 15/01/2024 Great service! 5 2 16/01/2024 Poor service 2 Average 3 17/01/2024 3 experience 4 18/01/2024 Excellent service 4 5 19/01/2024 Great service! 5 6 20/01/2024 Poor service 7 21/01/2024 Excellent service 4 Lab Activity Instructions: Open the Excel file and clean the data using functions like TRIM(), CLEAN(), and SUBSTITUTE(). Standardize the feedback text (e.g., remove extra spaces, correct inconsistencies). Normalize the rating scale if necessary. Lab Activity 3: Handling Missing Data and Outliers Dataset: Employee Performance (CSV) Description: This dataset includes employee performance metrics with some missing values and potential outliers. Students will practice handling missing data and identifying outliers. 18 IT INST3 – Data Science Analytics Chapter II Subject Teacher: Edward B. Panganiban, Ph.D. Example Data: EmployeeID Age Department PerformanceScore E001 29 Sales 85 E002 34 HR 78 E003 28 Sales E004 45 IT 92 E005 31 HR 80 E006 29 Sales 115 E007 IT 88 E008 40 HR 72 E009 35 IT 89 E010 50 Sales 105 Import the CSV file into Excel. Identify missing data and decide on appropriate methods for imputation (mean, median, etc.). Use a boxplot to visualize potential outliers in the PerformanceScore column. Handle outliers based on the chosen method (e.g., transformation or removal). 19