AWS S3 Quiz

Amazon S3 Overview

• Amazon S3 is an object storage service that allows storing and retrieving any amount of data from anywhere on the Internet.
• It's a simple storage service that offers an extremely durable, highly available, and infinitely scalable data storage infrastructure at very low costs.

Key Features

• Distributed architecture with objects redundantly stored on multiple devices across multiple facilities (AZs) in an Amazon S3 region.
• Simple key-based object store with keys that can be any string and can be constructed to mimic hierarchical attributes.
• Supports S3 Object Tagging to organize data across all S3 buckets and/or prefixes.
• Provides a simple, standards-based REST web services interface that is designed to work with any Internet-development toolkit.

Storage Details

• Files can be from 0 bytes to 5TB in size.
• The largest object that can be uploaded in a single PUT is 5 gigabytes.
• Objects larger than 100 megabytes require the Multipart Upload capability.
• Updates to an object are atomic, ensuring either the new object or the old one is retrieved, never partial or corrupt data.

Access Control

• Access to buckets and objects can be granted to AWS accounts, IAM users, and federated users.
• Supports four mechanisms for controlling access to Amazon S3 resources: IAM permissions, Bucket Policies, ACLs, and Query String Authentication.
• Requester pays function causes the requester to pay (removes anonymous access) and provides time-limited access to objects.

Data Consistency

• Provides read after write consistency for PUTS of new objects.
• Provides eventual consistency for overwrite PUTS and DELETES (takes time to propagate).

Additional Capabilities

• Supports unlimited storage with automatic scaling to high request rates.
• Allows for event notifications for specific actions, sending alerts or triggering actions.
• Supports cross-region replication with versioning enabled on both source and destination buckets.
• Enables fast, easy, and secure transfers of files over long distances with Amazon S3 Transfer Acceleration.

Storage Classes

• Offers six storage classes: S3 Standard, S3 Infrequent Access, S3 One Zone-Infrequent Access, S3 Glacier, S3 Glacier Deep Archive, and S3 Intelligent-Tiering.

Bucket Management

• Buckets are region-specific and have a unique namespace.
• Bucket names can be up to 63 characters and must be DNS-compliant.
• Supports up to 100 buckets per account by default, and objects can be stored in folders within buckets.
• Bucket ownership is not transferable, and bucket names cannot be changed after creation.

Objects and Sub-Resources

• Each object is stored and retrieved by a unique key (ID or name).
• Sub-resources are subordinate to objects, including object tags, access control lists (ACLs), and metadata.

Security and Encryption

• Supports server-side encryption options: SSE-S3, SSE-KMS, and SSE-C.
• Enables client-side encryption using customer-provided encryption keys.
• Supports encryption and decryption of data in transit using SSL/TLS.

Event Notifications

• Supports event notifications for specific actions, such as PUTs, POSTs, COPYs, or DELETEs.
• Enables notifications to be sent to Amazon SNS, Amazon SQS, or AWS Lambda.

Metrics and Analytics

• Supports Amazon S3 CloudWatch metrics for request and storage data.
• Enables configuration of filters for metrics using a prefix or object tag.

Cross-Region Replication

• Automatically replicates data across AWS Regions with Cross-Region Replication (CRR).

• Requires versioning to be enabled on both the source and destination buckets.

• Supports automatic, asynchronous copying of objects between buckets in different regions.### Cross-Region Replication (CRR)

• Source and destination buckets must be in different regions.

• Replication is 1:1 (one source bucket to one destination bucket).

• You can configure separate S3 Lifecycle rules on the source and destination buckets.

• You can replicate KMS-encrypted objects by providing a destination KMS key in your replication configuration.

• You can set up CRR across AWS accounts to store your replicated data in a different account in the target region.

• To activate CRR, you need to configure the replication on the source bucket.

• Replicas will be exact replicas and share the same key names and metadata.

• You can specify a different storage class (by default, the source storage class will be used).

• AWS S3 will encrypt data in-transit with SSL.

Same Region Replication (SRR)

• SRR allows you to replicate objects to a destination bucket within the same region as the source bucket.
• Replication is automatic and asynchronous.
• New objects uploaded to an Amazon S3 bucket are configured for replication at the bucket, prefix, or object tag levels.
• Replicated objects can be owned by the same AWS account as the original copy or by different accounts, to protect from accidental deletion.
• Replication can be to any Amazon S3 storage class, including S3 Glacier and S3 Glacier Deep Archive to create backups and long-term archives.

S3 Analytics

• S3 Analytics allows you to run analytics on data stored on Amazon S3.
• This includes data lakes, IoT streaming data, machine learning, and artificial intelligence.
• You can use S3 Inventory to audit and report on the replication and encryption status of your objects for business, compliance, and regulatory needs.

S3 Inventory

• S3 Inventory provides comma-separated values (CSV), Apache optimized row columnar (ORC) or Apache Parquet (Parquet) output files that list your objects and their corresponding metadata on a daily or weekly basis for an S3 bucket or a shared prefix.
• You can use S3 Inventory to audit and report on the replication and encryption status of your objects for business, compliance, and regulatory needs.

Monitoring and Reporting

• Amazon CloudWatch metrics for Amazon S3 can help you understand and improve the performance of applications that use Amazon S3.
• You can use CloudWatch with Amazon S3 to monitor and report on performance.

Logging and Auditing

• You can record the actions taken by users, roles, or AWS services on Amazon S3 resources and maintain log records for auditing and compliance purposes.
• You can use Amazon S3 server access logging, AWS CloudTrail logs, or a combination of both to log and audit actions on Amazon S3 resources.

S3 Performance Guidelines

• Measure Performance: Look at network throughput, CPU, and DRAM requirements when optimizing performance.
• Scale Storage Connections Horizontally: Issue multiple concurrent requests to Amazon S3 and spread them over separate connections to maximize the accessible bandwidth from Amazon S3.
• Use Byte-Range Fetches: Fetch a byte-range from an object using the Range HTTP header in a GET Object request to achieve higher aggregate throughput.
• Retry Requests for Latency-Sensitive Applications: Aggressive timeouts and retries help drive consistent latency.
• Combine Amazon S3 (Storage) and Amazon EC2 (Compute) in the Same AWS Region: Access the bucket from Amazon EC2 instances in the same AWS Region when possible to reduce network latency and data transfer costs.
• Use Amazon S3 Transfer Acceleration to Minimize Latency Caused by Distance: Use Amazon S3 Transfer Acceleration to minimize latency caused by distance between the client and an S3 bucket.

Glacier

• Glacier is an archiving storage solution for infrequently accessed data.
• There are three storage tiers: S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval (Formerly S3 Glacier), and Amazon S3 Glacier Deep Archive (S3 Glacier Deep Archive).
• The key difference between the top tiers is that Deep Archive is lower cost, but retrieval times are much longer (12 hours).
• Archived objects are not available for real-time access, and you need to submit a retrieval request.
• Glacier must complete a job before you can get its output.
• Requested archival data is copied to S3 One Zone-IA.
• Following retrieval, you have 24 hours to download your data.
• You cannot specify Glacier as the storage class at the time you create an object.
• Glacier is designed to sustain the loss of two facilities.
• Glacier automatically encrypts data at rest using AES 256 symmetric keys and supports secure transfer of data over SSL.
• Glacier may not be available in all AWS regions.
• Glacier objects are visible through S3 only (not Glacier directly).
• Glacier does not archive object metadata; you need to maintain a client-side database to maintain this information.
• Archives can be 1 byte up to 40TB.
• Glacier file archives of 1 byte – 4 GB can be performed in a single operation.
• Glacier file archives from 100MB up to 40TB can be uploaded to Glacier using the multipart upload API.
• Uploading archives is synchronous, and downloading archives is asynchronous.
• The contents of an archive that has been uploaded cannot be modified.
• You can upload data to Glacier using the CLI, SDKs, or APIs – you cannot use the AWS Console.
• Glacier adds 32-40KB (indexing and archive metadata) to each object when transitioning from other classes using lifecycle policies.
• AWS recommends that if you have lots of small objects, they are combined in an archive (e.g., zip file) before uploading.
• A description can be added to archives, but no other metadata can be added.
• Glacier archive IDs are added upon upload and are unique for each upload.

Archive Retrieval

• You can retrieve parts of an archive.
• When data is retrieved, it is copied to S3, and the archive remains in Glacier, and the storage class does not change.
• AWS SNS can send notifications when retrieval jobs are complete.
• Retrieved data is available for 24 hours by default (can be changed).
• To retrieve specific objects within an archive, you can specify the byte range (Range) in the HTTP GET request (need to maintain a DB of byte ranges).

Glacier Charges

• There is no charge for data transfer between EC2 and Glacier in the same region.
• There is a charge if you delete data within 90 days.
• When you restore, you pay for:

Amazon S3 Overview

• Amazon S3 is an object storage service that allows storing and retrieving any amount of data from anywhere on the Internet.
• It's a simple storage service that offers an extremely durable, highly available, and infinitely scalable data storage infrastructure at very low costs.

Key Features

• Distributed architecture with objects redundantly stored on multiple devices across multiple facilities (AZs) in an Amazon S3 region.
• Simple key-based object store with keys that can be any string and can be constructed to mimic hierarchical attributes.
• Supports S3 Object Tagging to organize data across all S3 buckets and/or prefixes.
• Provides a simple, standards-based REST web services interface that is designed to work with any Internet-development toolkit.

Storage Details

• Files can be from 0 bytes to 5TB in size.
• The largest object that can be uploaded in a single PUT is 5 gigabytes.
• Objects larger than 100 megabytes require the Multipart Upload capability.
• Updates to an object are atomic, ensuring either the new object or the old one is retrieved, never partial or corrupt data.

Access Control

• Access to buckets and objects can be granted to AWS accounts, IAM users, and federated users.
• Supports four mechanisms for controlling access to Amazon S3 resources: IAM permissions, Bucket Policies, ACLs, and Query String Authentication.
• Requester pays function causes the requester to pay (removes anonymous access) and provides time-limited access to objects.

Data Consistency

• Provides read after write consistency for PUTS of new objects.
• Provides eventual consistency for overwrite PUTS and DELETES (takes time to propagate).

Additional Capabilities

• Supports unlimited storage with automatic scaling to high request rates.
• Allows for event notifications for specific actions, sending alerts or triggering actions.
• Supports cross-region replication with versioning enabled on both source and destination buckets.
• Enables fast, easy, and secure transfers of files over long distances with Amazon S3 Transfer Acceleration.

Storage Classes

• Offers six storage classes: S3 Standard, S3 Infrequent Access, S3 One Zone-Infrequent Access, S3 Glacier, S3 Glacier Deep Archive, and S3 Intelligent-Tiering.

Bucket Management

• Buckets are region-specific and have a unique namespace.
• Bucket names can be up to 63 characters and must be DNS-compliant.
• Supports up to 100 buckets per account by default, and objects can be stored in folders within buckets.
• Bucket ownership is not transferable, and bucket names cannot be changed after creation.

Objects and Sub-Resources

• Each object is stored and retrieved by a unique key (ID or name).
• Sub-resources are subordinate to objects, including object tags, access control lists (ACLs), and metadata.

Security and Encryption

• Supports server-side encryption options: SSE-S3, SSE-KMS, and SSE-C.
• Enables client-side encryption using customer-provided encryption keys.
• Supports encryption and decryption of data in transit using SSL/TLS.

Event Notifications

• Supports event notifications for specific actions, such as PUTs, POSTs, COPYs, or DELETEs.
• Enables notifications to be sent to Amazon SNS, Amazon SQS, or AWS Lambda.

Metrics and Analytics

• Supports Amazon S3 CloudWatch metrics for request and storage data.
• Enables configuration of filters for metrics using a prefix or object tag.

Cross-Region Replication

• Automatically replicates data across AWS Regions with Cross-Region Replication (CRR).

• Requires versioning to be enabled on both the source and destination buckets.

• Supports automatic, asynchronous copying of objects between buckets in different regions.### Cross-Region Replication (CRR)

• Source and destination buckets must be in different regions.

• Replication is 1:1 (one source bucket to one destination bucket).

• You can configure separate S3 Lifecycle rules on the source and destination buckets.

• You can replicate KMS-encrypted objects by providing a destination KMS key in your replication configuration.

• You can set up CRR across AWS accounts to store your replicated data in a different account in the target region.

• To activate CRR, you need to configure the replication on the source bucket.

• Replicas will be exact replicas and share the same key names and metadata.

• You can specify a different storage class (by default, the source storage class will be used).

• AWS S3 will encrypt data in-transit with SSL.

Same Region Replication (SRR)

• SRR allows you to replicate objects to a destination bucket within the same region as the source bucket.
• Replication is automatic and asynchronous.
• New objects uploaded to an Amazon S3 bucket are configured for replication at the bucket, prefix, or object tag levels.
• Replicated objects can be owned by the same AWS account as the original copy or by different accounts, to protect from accidental deletion.
• Replication can be to any Amazon S3 storage class, including S3 Glacier and S3 Glacier Deep Archive to create backups and long-term archives.

S3 Analytics

• S3 Analytics allows you to run analytics on data stored on Amazon S3.
• This includes data lakes, IoT streaming data, machine learning, and artificial intelligence.
• You can use S3 Inventory to audit and report on the replication and encryption status of your objects for business, compliance, and regulatory needs.

S3 Inventory

• S3 Inventory provides comma-separated values (CSV), Apache optimized row columnar (ORC) or Apache Parquet (Parquet) output files that list your objects and their corresponding metadata on a daily or weekly basis for an S3 bucket or a shared prefix.
• You can use S3 Inventory to audit and report on the replication and encryption status of your objects for business, compliance, and regulatory needs.

Monitoring and Reporting

• Amazon CloudWatch metrics for Amazon S3 can help you understand and improve the performance of applications that use Amazon S3.
• You can use CloudWatch with Amazon S3 to monitor and report on performance.

Logging and Auditing

• You can record the actions taken by users, roles, or AWS services on Amazon S3 resources and maintain log records for auditing and compliance purposes.
• You can use Amazon S3 server access logging, AWS CloudTrail logs, or a combination of both to log and audit actions on Amazon S3 resources.

S3 Performance Guidelines

• Measure Performance: Look at network throughput, CPU, and DRAM requirements when optimizing performance.
• Scale Storage Connections Horizontally: Issue multiple concurrent requests to Amazon S3 and spread them over separate connections to maximize the accessible bandwidth from Amazon S3.
• Use Byte-Range Fetches: Fetch a byte-range from an object using the Range HTTP header in a GET Object request to achieve higher aggregate throughput.
• Retry Requests for Latency-Sensitive Applications: Aggressive timeouts and retries help drive consistent latency.
• Combine Amazon S3 (Storage) and Amazon EC2 (Compute) in the Same AWS Region: Access the bucket from Amazon EC2 instances in the same AWS Region when possible to reduce network latency and data transfer costs.
• Use Amazon S3 Transfer Acceleration to Minimize Latency Caused by Distance: Use Amazon S3 Transfer Acceleration to minimize latency caused by distance between the client and an S3 bucket.

Glacier

• Glacier is an archiving storage solution for infrequently accessed data.
• There are three storage tiers: S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval (Formerly S3 Glacier), and Amazon S3 Glacier Deep Archive (S3 Glacier Deep Archive).
• The key difference between the top tiers is that Deep Archive is lower cost, but retrieval times are much longer (12 hours).
• Archived objects are not available for real-time access, and you need to submit a retrieval request.
• Glacier must complete a job before you can get its output.
• Requested archival data is copied to S3 One Zone-IA.
• Following retrieval, you have 24 hours to download your data.
• You cannot specify Glacier as the storage class at the time you create an object.
• Glacier is designed to sustain the loss of two facilities.
• Glacier automatically encrypts data at rest using AES 256 symmetric keys and supports secure transfer of data over SSL.
• Glacier may not be available in all AWS regions.
• Glacier objects are visible through S3 only (not Glacier directly).
• Glacier does not archive object metadata; you need to maintain a client-side database to maintain this information.
• Archives can be 1 byte up to 40TB.
• Glacier file archives of 1 byte – 4 GB can be performed in a single operation.
• Glacier file archives from 100MB up to 40TB can be uploaded to Glacier using the multipart upload API.
• Uploading archives is synchronous, and downloading archives is asynchronous.
• The contents of an archive that has been uploaded cannot be modified.
• You can upload data to Glacier using the CLI, SDKs, or APIs – you cannot use the AWS Console.
• Glacier adds 32-40KB (indexing and archive metadata) to each object when transitioning from other classes using lifecycle policies.
• AWS recommends that if you have lots of small objects, they are combined in an archive (e.g., zip file) before uploading.
• A description can be added to archives, but no other metadata can be added.
• Glacier archive IDs are added upon upload and are unique for each upload.

Archive Retrieval

• You can retrieve parts of an archive.
• When data is retrieved, it is copied to S3, and the archive remains in Glacier, and the storage class does not change.
• AWS SNS can send notifications when retrieval jobs are complete.
• Retrieved data is available for 24 hours by default (can be changed).
• To retrieve specific objects within an archive, you can specify the byte range (Range) in the HTTP GET request (need to maintain a DB of byte ranges).

Glacier Charges

• There is no charge for data transfer between EC2 and Glacier in the same region.
• There is a charge if you delete data within 90 days.
• When you restore, you pay for: