Understanding Data and Knowledge

Data, Information, Knowledge, and Wisdom

• Data is a single observation or data point. A collection of data is a data set.
• Data in statistics is a collection of data or data set.
• Information is data that is processed and organized to be meaningful and useful for a specific purpose.
• Knowledge involves understanding, integrating, and applying information, principles and patterns.
• Wisdom is accumulated knowledge used to apply concepts to new situations or problems.

Types of Data

• Qualitative data (categorical data) describes categories and isn't numerical. Arithmetic can't be performed on it.
• Quantitative data (numerical data) can be measured and counted. Arithmetic operations are possible. -Nominal: labels, no order or quantitative value (e.g. hair color, gender). -Ordinal: has a natural order (e.g. ranking, education levels). -Discrete: whole numbers, finite values (e.g., total students in a class). -Continuous: fractional numbers, infinite values within a range (e.g., height).

Data Quality Dimensions

• Accuracy: How closely data reflects the real-world object or event.
• Completeness: How much of the expected data is available.
• Consistency: Uniformity of data across applications and networks.
• Validity: Whether data conforms to the organization's specified rules and conforms to the acceptable format.
• Uniqueness: No duplications are present.
• Timeliness: Data is available when needed in a timely fashion.

Big Data

• Enormous amounts of data generated and collected by organizations.

• Includes data from various sources (social media, sensors, transactions, etc.).

• Characteristics of Big Data (5Vs):

  • Volume: massive amounts of data generated.
  • Velocity: data arrives rapidly.
  • Variety: data comes in different forms (structured, unstructured, semi-structured).
  • Veracity: how accurate and reliable the data is.
  • Value: usefulness of the data to an enterprise.

Big Data Sources

• Logs
• Transactional Data
• Social Media Data
• Sensor Data
• Clickstream Data
• Surveillance Data
• Healthcare Data
• Network Data

Structured Data

• Data that conforms to a data model or schema.
• Often stored in tabular form (e.g., in databases).

Unstructured Data

• Data that does not conform to a data model or schema.
• Mostly qualitative data.
• Examples include text, images, audio, videos.

Semi-structured Data

• Data with some structure and consistency, but it's not relational.
• Commonly stored in files with text (e.g., XML, JSON).

Metadata

• Data about data.
• Describes data's characteristics, structure, and provenance.
• Crucial for managing and analyzing big data.

Data Analytics in Business

• Descriptive Analysis: Summarizing past data to understand trends.

• Diagnostic Analysis: Discovering the reason for specific events.

• Predictive Analysis: Forecasting future trends.

• Prescriptive Analysis: Suggesting actions to take to achieve desired outcomes.

• Applications in Business:

  • Production and Inventory Management
  • Sales and Operations Management
  • Price Setting and Optimization
  • Finance and Investments
  • Marketing Research
  • Human Resource Management.