Data Mining and Analytics Lec.1

• Data Mining and Analytics are covered in Lec.1.

Why Data Mining

• The explosive growth of data, from terabytes to petabytes, necessitates data mining.
• Data collection and availability also contribute to the need for data mining.
• Automated data collection tools, database systems, the Web, and a computerized society all drive the need for data mining.
• Major sources of abundant data include business, science, and society.
  • In business, these sources include the Web, e-commerce, transactions, and stocks.
  • In science, these sources include remote sensing, bioinformatics, and scientific simulations.
  • In society, these sources include news and digital cameras.
• Necessity is the mother of invention is a relevant quote to data mining.
• Data mining is an automated analysis of massive data sets.

What is Data Mining

• Data mining involves searching for relationships and global patterns within large databases that are hidden within vast amounts of data.
• These relationships represent valuable knowledge about the database and the objects it contains.
• Knowledge discovery in databases (KDD) is an alternative name for data mining.
• Data mining is seen as the core process of knowledge discovery.

Knowledge Process

• Data cleaning is the removal of noise and inconsistent data.
• Data integration combines data from multiple sources.
• Data selection retrieves relevant data for analysis.
• Data transformation converts data into an appropriate form for data mining.
• Data mining extracts data patterns using intelligent methods and is considered an essential process.
• Pattern evaluation identifies truly interesting patterns representing knowledge which is based on measures of interestingness.
• Knowledge presentation uses visualization and knowledge representation techniques.
• Data mining is often used as a synonym for knowledge discovery from data, or KDD.
• Steps 1 to 4 of the knowledge process are forms of data preprocessing.
• Data mining is essential because it uncovers hidden patterns for evaluation.
• A typical data mining system may have components such as databases, data warehouses, the World Wide Web, information repositories, database servers, data mining engines, pattern evaluation models, and user interfaces.
• Data mining employs computer learning techniques to automatically analyze and extract knowledge from data within a database.
• Data mining focuses on exploration and analysis of large quantities of data to discover meaningful patterns and rules.

What is a Data Warehouse

• A data warehouse is a single, complete, and consistent data store obtained from different sources.
• It is a data store made available to end users in a context to help understanding and use.
• Data warehousing is the entire process of data extraction, transformation, and loading to the warehouse.
• It also allows the access of the data by end users and applications.
• Data warehouses are collections of data created to support decision-making applications.
• A data warehouse is a historical decision support database that is maintained separately from an organization's operational database.
• Data transfer from the operational database to the warehouse is ongoing.
• Data warehousing is the process of constructing and using data warehouses.

Data Mining vs Data Warehousing

• Data mining is a method for comparing large amounts of data to find patterns and is normally used for models and forecasting.
• Data warehousing extracts data from different sources, cleans it, and stores in a warehouse.

Potential Applications

• Data analysis and decision support can be achieved.
• Data mining is helpful for market analysis and management.
  • This includes target marketing, customer relationship management (CRM), market basket analysis, and market segmentation.
• Risk analysis and management are also potential applications.
• Data mining is useful for forecasting, customer retention, quality control, and competitive analysis.
• It is also useful for fraud detection and detection of unusual patterns (outliers).
• Other applications include:
  • Text mining (news groups, email, documents) and Web mining.
  • Stream data mining.
  • Bioinformatics and bio-data analysis.