Data Wrangling and Storage Technologies

Study Notes

When is data storage typically required in data wrangling?

When external datasets are acquired
When data is manipulated to be suitable for analysis
When data is processed via ETL (Extract, Transform, Load) activity

Big data storage technologies

Clusters: A collection of servers, usually with identical hardware, connected via a network to function as a single unit.
File Systems and Distributed File Systems: Methods for storing and organizing data on storage devices like flash drives, DVDs, and hard drives.
- A file is the smallest unit of storage within a file system.
NoSQL: A non-relational database management system.
Sharding: A database technique to distribute data across multiple servers.
Replication: Duplicating data across multiple servers for redundancy and availability.
Sharding and Replication: Combining sharding and replication for improved performance and fault tolerance.
CAP Theorem: A theoretical framework describing the trade-offs in distributed systems between Consistency, Availability, and Partition tolerance.
ACID (Atomicity, Consistency, Isolation, Durability): Properties of database transactions ensuring reliability in traditional relational databases.
BASE (Basic Availability, Soft State, Eventually Consistent): Properties of database transactions suitable for less demanding applications when consistency is not a stringent requirement.

Description

Explore the critical concepts of data storage in data wrangling, including when it's typically needed and the various big data storage technologies available. This quiz will cover clusters, NoSQL databases, sharding, and the CAP theorem, providing a comprehensive understanding of data management strategies.