Untitled Quiz

Study Notes

Big data is data whose scale, diversity and complexity necessitate new architecture, techniques, algorithms and analytics to manage and extract value and hidden knowledge from it.
Big data is characterized by the 3Vs: volume, velocity, and variety.
Volume refers to the sheer amount of data.
Velocity describes the speed at which data is generated and needs to be processed.
Variety emphasizes the different formats and types of data (structured, unstructured, semi-structured).

Data mining is the process of sorting through large data sets to identify patterns and relationships.
These patterns help solve business problems through data analysis and help predict future trends, thus aiding informed business decisions.
The process of data mining involves several steps: collection, understanding, preparation, modeling, and evaluation.

Data warehousing is a system of managing very large amounts of data and extracting value from them.
Data streams are continuous flows of data needing to be processed rapidly.
The arrival rate of data streams makes storage capacity a challenge.
"Real-time" processing is required for data streams, necessitating effective decision-making.
Window models are used in data stream processing.

Data mining tasks include classification, clustering, association rule discovery, sequential pattern discovery, regression, deviation detection, and collaborative filtering.

Text mining applies data mining to textual documents, such as clustering web pages to find related pages or classify them into a web directory.
Graph mining deals with graph data.

New architecture, algorithms, and techniques are needed to handle the big data boom.
Experts with technical skills are crucial to successfully use the new technologies and deal with big data.
Technologies like Hadoop, Hive, Vertica, MapReduce, and others are used in big data applications.