Big Data Storage Concepts

The size of the data itself becomes a challenge due to limited storage capacity and processing power.

The time it takes to read or write data from a storage device (like a hard drive).

Distributing data across multiple computers for faster processing and storage.

A storage unit consisting of multiple hard drives in a single server for increased storage capacity.

Using multiple servers to store and process data for faster access and better scalability.

A storage system where data is spread across multiple servers, allowing for efficient data access, fault tolerance, and scalability.

The ability of a system to handle increasing workloads and data volumes gracefully.

The ability of a system to continue functioning if a part of the system fails.

Ensures only data that conforms to database rules (schema) can be written.

Guarantees that a transaction's effects are not visible until it's fully complete.

Ensures that changes made by a transaction are permanent, even after a system failure.

Ensures a database is always available, even if it's split (partitioned) due to network issues.

Allows the database to be in a temporary inconsistent state while data is being updated.

Means the database eventually becomes consistent once updates are propagated to all nodes.

Prioritizes consistency (accurate data) over availability (being online).

Prioritizes availability (being online) over immediate consistency (accurate data).

A database design principle that relaxes strict consistency rules for better availability in distributed systems.

A set of properties (Atomicity, Consistency, Isolation, Durability) that describe how a database guarantees data integrity during transactions.

A tightly coupled group of servers (nodes) with identical hardware, working as a single unit.

A file system that stores large files across multiple nodes in a cluster.

A database that manages and stores data in tables, with rows representing records and columns representing data fields.

A database that is non-relational, highly scalable, fault-tolerant, and designed for semi-structured or unstructured data.

The process of dividing a large dataset into smaller, independent parts (shards) for easier management and scalability.

Storing multiple copies (replicas) of data across multiple nodes in a cluster.

Replication where all data is written to a master node and then replicated to multiple slave nodes for read operations.

A theorem that explores the trade-offs between consistency, availability, and partition tolerance in distributed systems.

A set of properties that guarantee data consistency and reliability in database transactions.

A set of properties that focus on eventual consistency and availability, commonly used in NoSQL databases.

A unit of work performed against a database, treated as a single, coherent operation.

A language used for querying and managing relational databases.

A type of NoSQL database that uses key-value pairs for storing and retrieving data.

A type of NoSQL database that stores data in JSON-like documents.

A type of NoSQL database that uses wide rows with columns representing different attributes.

A type of NoSQL database designed for managing graph-structured data, representing relationships.

A replication strategy where a single master node handles all writes, and multiple slave nodes handle reads. Changes are replicated from the master to slaves, ensuring data consistency.

In master-slave replication, if the master node fails, reads can still be served by slave nodes. However, writes are not supported until a new master is established.

A potential issue in master-slave replication where a read from a slave might return outdated data before the latest changes propagate from the master.

A replication strategy where all nodes are peers, capable of both reads and writes. All changes are replicated to every peer.

A potential read inconsistency issue in peer-to-peer replication where a node might not have the latest data yet, leading to outdated information.

A potential write inconsistency issue in peer-to-peer replication where simultaneous updates on different peers might lead to conflicting data.

A proactive approach to handling write inconsistency where locks are used to ensure only one update happens at a time, guaranteeing consistency. However, this approach can impact database availability.

A reactive approach to handle write inconsistency that allows updates to occur even if inconsistent data is present. Eventually, all updates propagate, resolving the inconsistencies.

Sharding and replication can work together to enhance data management.

Each node acts as a master and a slave for different shards. Writes to a shard are handled by its designated master, while reads can be served by any node containing the shard.

Each node contains replicas of different shards, enabling parallel writes and reads across shards.

A theorem stating that a distributed system can only satisfy two out of three properties: Consistency, Availability, and Partition Tolerance.

Ensures that all transactions are completed successfully (commit) or no effect is made (rollback), preventing partial updates.

Guarantees that a transaction leaves the data in a valid and consistent state, adhering to database rules.

A transaction's changes are isolated from other concurrent transactions, ensuring a single transaction's effects are complete before others can see them.

Ensures that committed transactions are permanently recorded even in the event of system failure.