Supervised Learning Quiz

Introduction to Data Science

• Data science is an interdisciplinary field focused on extracting knowledge from datasets, often massive in size.
• It involves data analysis, data preparation for analysis, and presenting findings to inform high-level decisions.
• Data science incorporates skills from computer science, mathematics, statistics, information visualization, and business.
• It's a deep study of extracting meaningful insights from raw, structured, and unstructured data using the scientific method, different technologies, and algorithms.
• Data science uses powerful hardware, programming systems, and efficient algorithms to solve data-related problems and is the future of artificial intelligence.

Definition of Data science

• Data science is an interdisciplinary field focused on extracting knowledge from vast datasets.
• It encompasses data analysis, preparation, and presentation of findings to inform organizational decisions.
• Data science combines skills from computer science, mathematics, statistics, information visualization, and business.
• It profoundly studies data to produce significant insights from unstructured or structured data.

Big Data and Data Science hype

• Big data refers to large volumes of complex, varied data formatted data generated at a fast pace, which cannot be processed by traditional systems.
• It's often formatted in a manner that traditional applications can't process efficiently. Big data involves processing raw data, which isn't aggregated previously.
• Big data is used to analyze insights, leading to better business decisions and strategic moves.
• Social media, sensor data, customer feedback, IoT devices are crucial sources of big data.

IoT Appliances

• Internet-connected devices (IoT appliances) create data for smart functionality.
• Examples include smart TVs, washing machines, coffee machines, and air conditioners, which create data by using sensors located in them.
• Data transfer between these connected devices generates data.

E-commerce

• E-commerce transactions, including payments via credit/debit cards etc., create significant data.
• Business transactions, banking, and stock markets are crucial sources of big data.

Global Positioning System (GPS)

• GPS technology in vehicles monitors vehicle movement to shorten routes, minimize fuel consumption, and optimize time.
• GPS systems produce vast amounts of data on vehicle location and movement.

Applications of Big Data

• Big data is used in financial services for customer analytics, compliance, and fraud prevention.
• Credit card companies, banks, and insurance firms use big data to improve services, prevent risks, and make informed decisions.

Big Data in Communications

• Telecommunication companies use big data to increase subscriber base and retain customers.
• Companies analyze customer-generated and machine-generated data for solutions to subscriber challenges.

Big Data for Retail

• Retailers (brick-and-mortar and online) analyze disparate data sources (weblogs, transactions, social media, and loyalty programs) for improved customer understanding.

Data Science

• Data science is an area that deals with the collection, processing, analysis, and utilization of data used in multiple fields of operation.
• Focuses on a detailed study and understanding of the process of Data.

Big Data

• Big data is a technique to collect, maintain, and process massive amounts of information, extracting valuable insights.

Getting Past the Hype

• Rachel's experience illustrates that data science skills are practical and crucial to job success.

Datafication

• Datafication transforms social behavior into quantified data, enabling real-time tracking and predictive analysis.
• This translates everyday actions into digital (numeric) data, for analysis.

The Data Science Landscape

• Data science encompasses the disciplines of analytics, statistics, and machine learning.
• Analytics generates insights from data through visualization tools, calculations, and manipulations.

Statistics.

• Statistics is a mathematical branch for organizing, interpreting numerical data in precise ways, known as descriptive statistics.
• It's also used for making predictions from limited sample information about larger populations, called inferential statistics.
• Descriptive statistics summarize data characteristics using measures like mean, median, mode, variance, and standard deviation.
• Inferential statistics uses sample data to draw conclusions about a population.

Artificial Intelligence (AI)

• A field of computer science focusing on creating intelligent machines capable of mimicking human-like thinking and decision-making.
• Knowledge and skills gained through experience enable making decisions.
• AI is about mimicking human intelligence and building machines that can think and act like humans.

Machine Learning

• Machine learning is a subset of AI focused on developing algorithms that allow computers to learn from data and past experiences, without explicit programming.
• Models are built from historical data to predict outputs. Model accuracy is affected by data size.
• Machine learning systems can learn from data and make predictions.

Supervised Learning

• In supervised learning, machines are trained using labeled data where input data is already tagged with the correct output.
• These models learn to predict outputs correctly given new data.
• Image classification, fraud detection, and spam filtering are example applications of supervised learning.

Unsupervised Learning

• Unsupervised learning trains models on unlabeled data that isn't categorized/classified beforehand.
• The model identifies hidden patterns, similarities, and differences in the data, and then categorizes/classifies data based on its characteristics.
• Unsupervised techniques used for grouping similar data points include k-means clustering.

Statistical inference

• Statistics is a branch of mathematics used for the collection, analysis, interpretation, and presentation of numerical data.
• The goal is to make accurate conclusions about larger populations using a smaller sample set of data.

Population and Samples

• The complete collection of all objects or measurements is the population.
• A sample is a subset of the population selected for a specific study.

Overfitting

• Overfitting is a modeling issue where a statistical model is overly accurate/consistent/fitted to the training data but doesn't generalize/perform well on new, unseen data.

Statistical Modeling

• Construction and evaluation of mathematical models that use data.

Types of Probability Distribution

• Probability distribution is a function assigning probabilities to different outcomes.
• Discrete probability distributions deal with variables with countable values and probabilities e.g., flipping a coin.
• Continuous probability distributions deal with variables with values in a continuous range, such as time duration.

Basics of R

• R is an open-source programming language used for statistical computing and data visualization.
• It has a vast range of libraries (packages) and is highly extensible allowing users to perform diverse tasks.

Why R Programming Language

• R is an open-source statistical software and programming language frequently used for data analysis and statistical computing.
• Features include a vast community, easy installation, and integration with other computing languages.

Data Types in R

• Integer, logical, numeric, character, complex, and raw are the basic data types in R programming.
• Using the class() function, one can determine the data type of a variable.

R Commands

• The readline() function in R allows users to input values from the keyboard in the console. The inputted value is stored as a string.
• Using as.integer() function, raw string inputs can be converted into integer data type.

R Installation

• There are steps involved to install R and RStudio on Windows and Linux operating systems.