Information Engineering Overview

Information Engineering

• Information engineering studies and processes information using modern technologies like computers and communications.
• It aims to determine the best methods for saving, organizing, accessing, and retrieving information in automated systems or websites.

Data

• Data represents facts about the world, uninterpreted and raw.
• Example: "The price of crude oil is $80 per barrel."

Information

• Information is data with context meaning added.
• Example: "The price of crude oil has risen from $70 to $80 per barrel."

Knowledge

• Knowledge involves adding purpose and generative action for creating new information.
• Example: "When crude oil prices go up by $10 per barrel, it's likely that petrol prices will rise by 2p per litre."

Data Mining

• Data mining is the process of extracting knowledge from large datasets.
• It's similar to gold mining (extracting gold from rocks/sand) rather than simply rock or sand mining.

Why Mine Data (Commercial Viewpoint)

• Vast amounts of data are collected and stored (e.g., web data, e-commerce purchases, bank/credit card transactions).
• Computers are becoming more powerful and cheaper.
• Competitive pressure is strong, demanding better, customized services.

Why Mine Data (Scientific Viewpoint)

• Data is collected and stored at high speeds (e.g., GB/hour) from various sources (remote sensors, telescopes, microarrays).
• Traditional techniques are often insufficient for handling this raw data volume.
• Data mining assists in classifying/segmenting data and formulating hypotheses.

Definition of Data Mining

• Data mining is the process of discovering patterns in data.
• The process should be primarily automatic or semi-automatic.
• Discovered patterns should be meaningful and lead to advantages (typically economic).
• Data is usually present in substantial quantities.

Data Mining Process (Alternative Definition)

• Data mining utilizes various data analysis methods to identify the unknown, unexpected, interesting, and relevant patterns and relationships.
• This can enable making accurate and valid predictions.
• Data mining is synonymous with Knowledge Discovery from Data (KDD).

Architecture of a Data Mining System

• A data mining system typically involves user interface, pattern evaluation, data mining engine, and knowledge base components.
• It also includes database/data warehouse server for data cleaning, integration, and selection along with data sources (databases, warehouses, World Wide Web, other repositories).

Data Warehouse

• A data warehouse is a system for reporting and data analysis.
• It's a core element of business intelligence.
• A data warehouse integrates data from various sources into a central repository.

Data Warehouse Components (Simplified)

• Data sources (e.g., OLTP servers, legacy systems, flat files)
• ETL tools (for extracting, transforming, and loading data)
• Data Staging Area (for interim data transformation)
• Data warehouse (for storing integrated data)
• Data marts (for focused data subsets)
• Decision support tools (e.g., data mining, OLAP, reporting, data visualization)

This quiz covers key concepts in information engineering, including the distinctions between data, information, and knowledge. It also touches on data mining and its significance in extracting valuable insights. Test your understanding of how modern technologies influence information processes.