Data and Information Concepts

Data vs. Information

• Data consists of raw facts that are not yet processed to reveal their meaning
• Information requires context to reveal the meaning of data
• Information can be measured, visualized and analyzed for a specific purpose
• Data is the base building block while Information is the second building block
• Data management involves the generation, storage, and retrieval
• Knowledge creation is enabled by information that's accurate, relevant, and timely for good decision-making

Data Repositories

• Data is stored in repositories in machine processable, searchable (using SQL), and human-understandable formats
• Data is stored practically in either a File System or a Database Management System (DBMS), or even both

Progression of Data

• Raw Fact is turned into Data
• Data is Processed into Information which is actionable
• Applying Knowledge to the Information creates Applied Knowledge
• Using WISDOM is the pinnacle

Data Object (Database Design Perspective)

• A Data Object can either be a Data Instance or a Data Schema
• A Raw fact is raw data on a Data Instance
• Structural or Unstructural data is found on the Data Instance
• Data instance example is name, ID number and gender of a particular person
• Data Record is defined on a Data Instance
• The Data Schema is the skeleton structure of data
• The Data Schema has properties or characteristics
• The Data Schema defines attributes

Database Architecture

• Users access data through different views (View 1, View 2, etc.) at the External or View Level
• A Logical Schema exists and is managed through External or Conceptual Mapping
• Beneath the Logical Schema is a Conceptual Level
• Below this is a Physical Schema which is managed using a Conceptual or internal mapping
• An Internal level is used to manage the Physical Schema
• At the bottom is the Database

Introducing the Database

• A database is a shared, integrated computer structure that stores end-user data and metadata
• End-user data is raw facts of interest
• Metadata is data about data, it provides descriptions of data characteristics and relationships
• A Database Management System (DBMS) is collection of programs
• The DBMS manages database structure, stores actual data, and secures/controls data access

From Data to Data Science

• The capability to extract useful knowledge from data is key for data science

Introduction to Data Science

• Data science involves principles, processes, and techniques for understanding phenomena
• Data science is an interdisciplinary field using scientific methods, processes, algorithms, and systems to:
  • Extract actionable knowledge/insights from noisy, structured/unstructured data
  • Apply this knowledge across various application domains
• It is a field of study concerned with:
  • Collection, cleaning, and anonymizing large quantities of relevant data
  • Solving real-life problems by analyzing data to initiate meaningful actions
• Data science uses the automated analysis of data

Data-Driven Decision Making

• Data-Driven Decision Making (DDD) involves decisions based on data analysis rather than intuition
• For example, instead of selecting ads based on experience, a marketer uses consumer data to select ads

Data Processing and Big Data

• Data engineering and processing are critical to support data science
• Data science often benefits from sophisticated data engineering and processing technologies

Big Data

• Datasets that are too large for traditional data processing systems are Big Data
• Big Data requires new processing technologies
• Big Data consists of key technologies like Hadoop, HDFS, NoSQL, MapReduce, MongoDB, Cassandra, PIG, HIVE, and HBASE
• These technologies work together to extract previously considered data

Data Analytics

• Analytics involves the discovery and communication of meaningful patterns in data
• Analytics is valuable in recorded information-rich areas
• Analytics relies on simultaneous applications in statistics, computer programming, and operations research to quantify performance.
• Data analytics is concerned with extracting actionable knowledge and insights from big data
• Hypothesis formulation based on experience-gathered conjectures and discovering variables' correlations enables this

KDD (Knowledge Discovery in Databases) Process

• Data undergoes several stages
• First is selection
• Then Preprocessing
• After this Data is Transformed
• Followed by Data Mining
• The final step is Interpretation/Evaluation

Levels of Analytics

• Analytics happens at several levels
• Descriptive Analytics answers what happened
• Diagnostic Analytics answers why it happened
• Predictive Analytics answers what will happen
• Prescriptive Analytics answer "Best" course of action

Business Questions

• Business questions can be basic
• Simple Stats looks at descriptive data
• Hypothesis Testing measures the variance from a given hypothesis
• Segmentation/Classification identifies customer characteristics
• Prediction helps determine profitability for the company

Applying Techniques

• Supervised Learning is one category with techniques like classification and regression included
• Unsupervised Learning is another category with techniques like Clustering and Dimension reduction included
• Examples of Supervised techniques are: kNN, Naïve Bayes, Logistic Regression, Support Vector Machines, Random Forests
• Examples of Unsupervised learning are: Clustering, Factor analysis, Latent Dirichlet Allocation
• Key note: Unsupervised Learning is often used inside a larger Supervised learning problem

Data Analytics Challenges

• Data Analytics can be measured based on Value
• Data can be grouped into Big Data, Processed Data, Reporting and Analytics
• Big Data gives access to Structured and Unstructured Data
• Processed Data allows for Indexed, Organized and Optimized Data
• Reporting gives an evaluation of what happened in the past through Identification of Patterns and Relationships
• Predictive Analytics gives Sets Of Potential Future Scenarios
• Prescriptive Analytics Automatically Prescribe and Take Action

Data Science Life Cycle

• The Data Science Life Cycle is cyclical in nature and split into 4
  • Data science analyzes experiments for findable, accessible, interoperable and reusable research outputs
  • Data science plans experiments and generate new hypotheses and select optimal parameters for experiments
  • Data science performs experiments with automated laboratories and data analysis and enables fast feedback

This quiz covers fundamental concepts related to data and information. It explores the relationship between data and information, the characteristics of good information, and the role of context in data interpretation. Questions also cover data management practices, data analytics goals, and levels of analytics.