Cloud Storage and Virtualization

• Cloud computing enables cloud storage and storage virtualization
• Prafullata Kiran Auradkar from the Department of Computer Science and Engineering created this material

Cloud Storage

• Cloud Storage is a complex service
• Variable-size files allow operations, like read, write, append, move, rename, lock, and unlock
• Fixed-size blocks allow read and write functions
• PC users experience a powerful interface with a hierarchical namespace, enabling operations like move, rename, append, truncate, compress, view, and delete files
• Actual storage devices are simple, HDD only knows how to read and write fixed-size data blocks
• Translation between file systems and storage devices is done by the operating system
• Many cloud services have a similar structure
• Users see a rich interface (shopping carts, product categories, searchable index, recommendations)
• Actual storage service is very simple; read/write 'blocks', similar to a giant hard disk
• Translation is done by the web service
• Applications can be thrown into the cloud, regardless of the server they land on
• Applications need to store/write or read data seamlessly
• Data should be accessible even if the VM migrates
• Cloud Storage provides the ability for an application to save data and files in an off-site location, accessible via the internet or a dedicated network connection
• Storage Virtualization supports access, utilization, availability, and necessary features for applications
• Terms like Private cloud, Public cloud, Hybrid Cloud, Internal cloud, and external cloud refer to storage location
• Cloud storage systems possess virtual storage pool and multi-tenancy capabilities
• Cloud Storage infrastructure includes hardware and software components
• Cloud storage is a data storage service model where data is maintained, managed, and backed up remotely over a network (typically the internet)
• Cloud storage providers keep data available, accessible, and the physical environment protected and running
• People and organizations buy or lease storage capacity from providers
• Object storage services like Amazon S3 and Microsoft Azure Storage, object storage software like Openstack Swift, and object storage systems like EMC Atmos and Hitachi Content Platform are hosted and deployed with cloud storage characteristics
• Cloud storage infrastructure consists of storage servers and representation servers connected through the internet and a LAN

Architecture of a Cloud Storage Platform

• User access layer: An authorized user logs into the cloud storage platform from any location via a standard public interface
• Data service layer: Deals directly with users, offering different application interfaces for data storage and space leasing
• Data management: Provides an upper layer with a unified public management interface for different services, like user management and security

Cloud Storage Services

• Cloud storage services offer pros such as lower cost, easier management, and enables sharing and access from anywhere. Cloud storage has cons like loss of control, no guarantees of data availability and provider failures
• Cloud storage results in a dramatic reduction in TCO (cuts storage cost by more than 10 times compared to block or file storage)
• Cloud storage leads to unlimited scalability, since built using distributed technologies, has unlimited scalability
• Adding or removing storage systems can be done seamlessly from the pool
• Storage virtualization decouples and abstracts the storage pool from its physical implementation, creating an elastic and unified storage pool
• Cloud storage uses a pay-as-you-go model
• Pay only for accessed data for a private cloud. Minimal cluster to start with, beyond which it is on-demand.
• Results in huge cost savings for the storage user
• Cloud storage offers universal access. Offers flexibility on the number of users and where those users are located
• Cloud storage is typically multi-tenant, supports centralized management, and results in higher storage utilization and lower costs
• Cloud storage exhibits data durability and availability. Runs on commodity hardware, but is supported by software layer
• Cloud storage has considerable usability

Cloud Storage Enablers

• Both storage and Compute resources are enabled for the cloud by virtualization
• Storage virtualization can be implemented in hardware or software
• Storage virtualization can be implemented in the Server, storage device, and the network carrying data
• Components like File Systems, Volume Manager, and Logical Volume Managers enable Storage virtualization in the Server
• Techniques like RAID and Logical Volume Management are also used with Storage virtualization in the storage device
• File systems, logical volume managers, and thin provisioning are enablers of cloud storage virtualization

File Systems

• Data is separated and grouped into pieces called a file
• A file system controls how data is stored and retrieved
• File systems provide a way to manage groups of information (files) and their names
• Without a file system, data in a storage medium would be one large body without identifying where one piece begins or ends
• Local file systems can be seen in this fashion. They can be used on many different types of storage media
• There are many different kinds of file systems
• Some file systems are used on local data storage devices
• Some file systems provide file access via a network interface and are responsible for arranging storage space
• Network file systems make local files and directories available over the LAN
• This allows users to work on common files across multiple locations
• Network File Systems support applications to share and access files remotely from several computers
• NFS functionality is a typical server setup to host files and directories, making them available across the network via the nfsd daemon
• The server administrator exports the directories and advertises them in a configuration file: NFS client requests to the exported directories with the mount command

Logical Volume Manager (LVM)

• The independent layer between the File System and the disk drives
• LVM allows the creation of partitions on the physical disk and the creation of physical volumes
• Physical volumes can be grouped into a volume group
• This volume group is then broken up into logical volumes
• File systems can be created on these Logical volumes and mounted

Thin Provisioning or Virtual Provisioning

• Capacity-on-demand from a shared storage pool
• Logical units presented to hosts have more capacity than physically allocated (The physical resources are thinly or virtually provisioned)
• Physical storage is allocated only when the host requires it
• Provisioning decisions are not bound by currently available storage
• Storage perceived by the application is larger than physically allocated storage