Cloud Computing Basics.pdf

Cloud Computing Basics Evolution of Technology Eras 1. PC Era o Computers are networked, and personal computers are connected through servers. 2. Mobile Era o Access to the Internet through mobile phones becomes common. 3. Advent of 5G o All...

Cloud Computing Basics Evolution of Technology Eras 1. PC Era o Computers are networked, and personal computers are connected through servers. 2. Mobile Era o Access to the Internet through mobile phones becomes common. 3. Advent of 5G o All devices, including computers, mobile phones, and intelligent terminals, can be connected seamlessly. 4. Internet of Everything (IoE) Era o Industries compete for ecosystems. Initially, ecosystems experience rapid changes but tend to stabilize over time. Challenges Faced by Conventional IT Architecture Long procurement periods lead to slow rollout of new business systems. Poor scalability of centralized architecture increases processing performance of a single node only. Traditional hardware devices exist independently, relying on software for reliability. Heterogeneous devices and vendors are difficult to manage. The performance of a single device is limited. Low utilization of devices results in high total costs. Enterprises Are Migrating to Cloud Architecture 1. Traditional IT Architecture o Requires significant power, physical space, and capital. Usually installed locally for enterprise or private use. 2. Virtualization Technology o Virtualization enables maximum utilization of physical hardware and simplifies software reconfiguration. 3. Cloud Transformation o Transforms enterprise data centers: ▪ From resource silos to resource pooling. ▪ From centralized to distributed architecture.  ▪ From dedicated hardware to software-defined storage (SDS). ▪ From manual handling to self-service and automatic service. ▪ From distributed statistics to unified metering. Definition of Cloud Computing Cloud Computing A model for enabling ubiquitous, convenient, and on-demand network access to a shared pool of configurable computing resources. Cloud A metaphor for networks and the Internet, referring to an abstraction of the infrastructure that underpins it. Computing Refers to computing services provided by powerful computers with various functionalities, resources, and storage. Cloud Computing Characteristics: Broad network access Fast and elastic scaling On-demand self-service Resource pooling Metered services Benefits of Cloud Computing Cloud computing allows enterprises to provide better products and attract more users. Examples of Cloud-Based Services: Cloud albums: Baidu Cloud, iCloud Shared Album Cloud music: NetEase Cloud Music, Kugou Music, Kuwo Music, Xiaomi Music Cloud video: Baidu Cloud, Tencent Cloud Video  Cloud documents: Youdao Note, Shimo Document E-Government Cloud Online Services Guidance Policies, bulletins, and processes are released via information guidance. Handling Fields in documents are auto-populated based on big data and AI technologies. Review AI technologies improve review efficiency and quality, reducing pressure on staff. Collaboration Robotic Process Automation (RPA) handles work items, collaborating with functional agencies. Public Cloud Development in China Public cloud is the dominant form of cloud computing in China. Public cloud vendors include: o Traditional telecom infrastructure carriers (e.g., China Mobile, China Unicom, China Telecom). o Local government cloud platforms. o Public cloud platforms from Internet giants (e.g., Alibaba Cloud, Tencent Cloud). o Internet Data Center (IDC) carriers (e.g., 21Vianet Group). o Enterprises with foreign technical backgrounds. Cloud Computing Models Deployment Models for Cloud Computing 1. Private Cloud A cloud infrastructure operated solely for a single organization. All data is kept within the organization's data center. 2. Public Cloud Cloud infrastructure is owned and operated by a service provider, offering cloud services to the public or enterprises. 3. Hybrid Cloud A combination of public and private cloud or on-premises resources. They remain distinct entities but are bound together, allowing migration of workloads across these environments as needed. Cloud Computing Service Models Infrastructure as a Service (IaaS): Provides infrastructure (servers, storage, networks, virtual resources). Users focus on system and application layers while the provider maintains related resources. Platform as a Service (PaaS): Provides infrastructure and application deployment environment (OS, middleware, software runtime). Users focus only on applications and data. Software as a Service (SaaS): Provides all resources, services, and maintenance. Users only need to utilize the applications. Cloud service advantages over conventional IT: IT devices are provided as services, selected on demand. Cloud services offer more flexibility and lower costs compared to the all-device procurement model. Benefits of Cloud Computing Resource Integration: Hardware resources are integrated into software, dynamically allocated to applications. This improves resource usage, supports auto-scaling, and optimizes efficiency. Centralized Data Storage: High-spec cloud data centers with automatic scheduling enhance centralized data storage, energy saving, emission reduction, and easier maintenance. Lower Costs & Higher Efficiency: Reduces operational costs and enhances productivity across various dimensions. Five Main Benefits of Cloud Computing On-demand self-service: Customers can deploy resources (processing, network, storage) based on actual requirements without provider interaction. Broad network access: Services accessible over the Internet from various devices (mobile phones, laptops, tablets). Resource pooling: Resources are centralized and dynamically allocated based on customer needs. Users are unaware of the exact resource locations (storage, processors, memory, bandwidth, virtual machines). Quick deployment & auto-scaling: Cloud services can be rapidly scaled to meet demand, allowing customers to rent and purchase resources as needed. Metered services: Billing is based on actual resource usage (CPU, memory, storage, bandwidth). Two billing models: pay-per-use or monthly/yearly subscriptions. Common Characteristics of Cloud Computing Massive scale: Cloud services centralize IT resources, making it distinct from conventional IT systems. Homogeneity: Standardization of resources, similar to how all electrical appliances use the same voltage and sockets. Virtualization: Divides computing units for better resource utilization. Separates software from hardware, allowing software to migrate freely across hardware. Elastic computing: IT resources are elastically provided, scaled up or down based on demand. Low-cost software: Cloud computing provides low-cost software, making IT more accessible to small startups and users with limited technical skills or budgets. Geographic distribution: Cloud data centers are geographically distributed, offering services globally with varying network performance across regions. Service orientation: Cloud computing is designed as a service model, focusing on delivering services to customers. Advanced security technology: Public clouds have diverse users, requiring advanced security technologies to protect data and services. Cloud Computing Technologies Compute Services: Provide the computing power needed for running services like websites, office software, and data analysis. Examples of compute cloud services: Virtual Machines (VMs) Containers Network Services: Provide resource connectivity and isolation. Examples include data center networks and campus networks. On the cloud, VMs use Virtual Private Clouds (VPCs), with a logical topology similar to traditional networks. Cloud Storage Services Block Storage: High performance, low latency. Suitable for high I/O service requirements. File Storage: Allows file sharing among multiple servers or departments. Object Storage: Flat, easy scale-out architecture. Suitable for massive data storage, cold data backup, and software repository. What is Virtualization? Definition: Virtualization refers to the process of creating multiple VMs that share the hardware resources of a physical server. VM Structure: A VM consists of disk files and description files, encapsulated in the same folder. Multiple VMs running on a server are encapsulated in separate folders, ensuring mutual isolation. These folders are stored in the underlying storage system, allowing VMs to run on a shared medium. Resource Management in Virtualization:  Abstracts physical resources like CPU, memory, disk space, and network adapters. Converts and presents them for segmentation and combination into multiple computer configurations. Virtualization Benefits: Breaks barriers between physical structures. Allows better utilization of hardware resources than the original physical configuration. Virtualization vs. Cloud Computing Virtualization: Transforms physical hardware into virtual resources. Powers cloud computing by abstracting physical resources into logical ones for flexible allocation. Provides scalable, distributed, and high-availability (HA) resources. Cloud Computing: An environment that delivers virtualized resources on-demand via the internet. Relies on virtualization to offer resources that users can access on demand. Main Features of Virtualization Partitioning: Multiple VMs run on one physical server, with resources allocated flexibly across VMs. Isolation: If one VM experiences an issue (e.g., viruses or failure), others on the same server remain unaffected. Encapsulation: VMs are stored as files, allowing easy migration by copying or moving these files. Independence: VMs can migrate to other physical servers without needing adjustments to their operating systems. Important Virtualization Concepts Hypervisor (Virtualization Software/VM Monitor): Software used to create and run VMs on physical servers (e.g., Xen, KVM). Guest OS: The operating system running inside a virtual machine. Guest Machine: The virtual machine (VM) itself. Host OS: The operating system of the physical machine. Host Machine: The physical server where the VMs run. Computing Resources Around Us Computing: The process of obtaining information, relying on resources to calculate data and extract information. Key Resources: CPU: The core computing and control unit of a computer, responsible for processing information and running programs. Memory: Stores data processed by the CPU and facilitates data exchange between memory and external storage. Disk: External storage used to store data. GPU: A microprocessor for image computations, commonly used in PCs, workstations, game consoles, and mobile devices. HUAWEI CLOUD Compute Services Elastic Cloud Server (ECS): A virtual machine (VM) on the cloud with virtual CPUs (vCPUs), memory, OS, and EVS disks. Can be used like a local PC or physical server once purchased. Auto Scaling (AS): Automatically adjusts the number of ECSs based on demand and AS policies. Scales compute resources according to changes in service load. Image Management Service (IMS): Manages the lifecycle of images, which are templates used to create servers or disks. Allows the creation of system/data disk images from servers or external image files, as well as full ECS images or ECS backups. Container Technology What is a Container?:  Containers are lightweight and portable units for application packaging, containing code and its dependencies. They run across various computing environments and are more efficient than virtual machines (VMs). Key Features of Containers: Package software into standardized units for easier development, migration, and deployment. Isolate compute, storage, and network resources. Instantly start, stop, deploy, and migrate applications. Enhance developer productivity and simplify operations and maintenance (O&M). Container Technology Development: 1. 1979: Introduced in UNIX as chroot for file system isolation. 2. 2000: FreeBSD Jails provided user, network, and resource isolation, along with individual IP addresses for jails. 3. 2005: OpenVZ added kernel virtualization, isolation, and resource management. 4. 2006: Google introduced Process Containers (later renamed cgroups) for resource limitation and isolation. 5. 2008: LXC was developed as the first complete Linux container manager using cgroups and Linux namespaces. 6. 2013: Docker was launched, offering a full ecosystem for container management. 7. 2014: CoreOS introduced rkt to improve container security. 8. 2016: Microsoft launched Hyper-V containers, providing both VM security and container lightweight performance. Difference Between Containers and Virtual Machines (VMs) Containers use OS-level virtualization, making them more efficient and portable compared to VMs, which virtualize hardware. Containers are faster, more agile, and lightweight, leading to better performance than VMs. Key benefit: Agile building and deployment of applications—creating container images is easier than VM images. Containers Abstraction Layer: Application level. Components: Packages up code and dependencies needed to run an application. Resource Usage: Containers share the same OS kernel but run as independent processes in user space.  Efficiency: o Takes up less space than VMs (often MB-sized images). o Requires less CPU and memory. o Can process more applications on the same hardware. Startup Speed: Faster startup compared to VMs. Virtual Machines (VMs) Abstraction Layer: Hardware level. Components: Each VM runs its own OS, applications, binaries, and libraries. Resource Usage: Multiple VMs can run on the same server using a hypervisor, but each VM operates independently with a full OS. Efficiency: o Takes up more space (VM images are typically hundreds of MB or more). o Requires more CPU and memory. o Typically fewer VMs can run on the same hardware compared to containers. Startup Speed: Slower startup due to the need to boot the entire OS. Container Image: Dedicated to running a specific service. Contains only the resources needed for the service, typically in the range of MBs. VM Image: Offers a full operating environment (including the OS kernel). Larger in size, typically hundreds of MBs or more. HUAWEI CLOUD Container Services: 1. Cloud Container Engine (CCE): o A high-performance, reliable service for managing containerized applications. o Supports Kubernetes applications, allowing easy container runtime setup on the cloud. o CCE Turbo Clusters: Uses Cloud Native 2.0 infrastructure to enhance computing, networking, and scheduling performance. 2. Cloud Container Instance (CCI): o A serverless container engine.  o Allows running containers without managing server clusters, making it easier to deploy. 3. SoftWare Repository for Container (SWR): o Manages the full lifecycle of container images, ensuring secure deployment. o Enables users to upload, download, and manage images through SWR Console, CLI, or APIs. o Can be used with CCE and CCI or as an independent container image repository. Basic Networking Concepts: 1. Broadcast vs. Unicast: o Unicast: One-to-one communication (like a private conversation). o Broadcast: One-to-many communication (like an announcement to a group). 2. Router: o Connects two or more networks and forwards data between them. o Operates at the third layer (network layer) of the OSI model. 3. Default Gateway: o A device that connects a local subnet to external networks, routing traffic when the destination is outside the local subnet. 4. Virtual Local Area Network (VLAN): o A logical grouping of devices or users, regardless of physical location, enabling communication as if they are on the same network segment. o Commonly used to isolate services within a network. Switches: 1. Layer 2 Switch: o Forwards electrical signals and creates exclusive routes between nodes in a LAN. o Often used for isolating networks with VLANs. 2. Layer 3 Switch: o Handles both switching and routing, designed for larger networks. o Combines the speed of Layer 2 switching with Layer 3 routing, making it more efficient than traditional routers. Basic Virtual Network Concepts: 1. Network Interface Card (NIC): o Connects devices to a network and can be used for expanding network capabilities. o Speed rates: Common NIC speeds include 100 Mbit/s, 1 Gbit/s, and 10 Gbit/s. 2. Virtual Network: o VMs on a physical machine may have different IP addresses but share the same physical NIC. o Virtual Switches: Used to map virtual NICs to physical NICs, facilitating communication between VMs and the external network. Common methods include: o Network Bridges o NAT (Network Address Translation) o Virtual Switches What Do Bridge and NAT Do? 1. Bridge: o A bridge connects different physical LANs at the data link layer (Layer 2 of the OSI model). o It forwards traffic from virtual machines (VMs) to physical network interface cards (NICs), routing data packets between VMs and external networks. o Virtual switches (like Open vSwitch) also have the same bridging function, isolating collision domains by mapping MAC addresses to specific ports. 2. NAT (Network Address Translation): o NAT translates internal private IP addresses to external public IP addresses, allowing VMs to communicate with external networks. o It conserves IP addresses by enabling multiple VMs to share a single public IP. o NAT also adds a layer of security by hiding internal IPs from external networks, protecting private networks from external attacks. What Does a Virtual Switch Do? A virtual switch forwards traffic between VMs and the external network using the physical NICs of the host server. It works similarly to a physical switch but in a virtualized environment. Common types of virtual switches include: 1. Open vSwitch (OVS):  o A software-based, open-source virtual switch. o Supports various networking protocols and integrates with open-source virtualization platforms. o Facilitates communication between VMs and external networks. 2. Enhanced vSwitch (EVS): o An enhanced version of OVS that uses Intel DPDK (Data Plane Development Kit) to improve I/O performance. o Processes data in user mode instead of kernel mode like OVS, resulting in faster data transmission. 3. Distributed Virtual Switch (DVS): o Connects VMs to each other and external networks, similar to a physical switch. o VMs communicate with external networks by connecting to the DVS, which links to the physical NIC of the host server through the DVS uplink. o Simplifies network architecture by reducing the need for additional physical network devices. HUAWEI CLOUD Network Services: 1. Virtual Private Cloud (VPC): o A private, isolated network on HUAWEI CLOUD, where users can manage IP addresses, subnets, and security groups. o VPCs allow users to assign Elastic IPs (EIPs) and allocate bandwidths to cloud servers for flexible, secure networking. 2. Public and Private NAT Gateways: o Public NAT Gateway: Provides SNAT (Source Network Address Translation) and DNAT (Destination Network Address Translation), allowing multiple cloud servers to share EIPs to access the Internet. o Private NAT Gateway: Translates internal private IP addresses within a VPC without exposing them to the Internet. 3. Elastic IP (EIP) Service: o Provides independent public IP addresses and bandwidth for cloud servers to access the Internet. o EIPs can be dynamically bound to and unbound from cloud resources like ECSs (Elastic Cloud Servers) and BMSs (Bare Metal Servers). Cloud storage involves using remote servers, typically hosted on the internet, to store and manage data. This model provides higher efficiency and lower costs compared to traditional storage methods. Here’s a breakdown: Storage Medium: Refers to technology and devices used to place, keep, and retrieve electronic data. In cloud storage, these technologies are optimized for scalability and cost-efficiency. Benefits: Cloud storage is becoming a preferred choice for individuals and businesses due to its flexibility, scalability, and cost-effectiveness. It offers easy access, data redundancy, and lower management overhead compared to traditional storage solutions. Mainstream Storage Types 1. Traditional Block Storage: o Description: Data is stored in fixed-sized blocks. Block storage connects disks directly to servers through buses, providing low latency. o Limitations: Limited capacity and bandwidth, poor performance for high-capacity requirements. o Modern Solution: Disk arrays with multiple controllers and disk enclosures enhance capacity and reliability. Block storage is typically used for high- performance applications like databases. 2. Shared File Storage: o Description: Enables multiple hosts to access the same data concurrently within the same data center or equipment room. o Use Cases: Ideal for scenarios requiring simultaneous access by multiple systems, such as HPC (High-Performance Computing) and office automation. 3. Object Storage: o Description: Uses a flat architecture for storing data, which simplifies scaling and management. It supports HTTP/HTTPS protocols and APIs. o Advantages: Handles large volumes of data efficiently with high reliability and durability. o Challenges: Requires application-level APIs, which may need redevelopment of traditional applications. o Use Cases: Suitable for big data, IoT (Internet of Things), backup, and archival. Storage Types 1. Block Storage: o Characteristics: Requires formatting and creating a file system before use. Provides the lowest latency and is suitable for mission-critical applications. o Ideal For: Databases, ERP systems.  2. File Storage: o Protocols: ▪ NFS (Network File System): Used for UNIX systems (mainly Linux). ▪ CIFS (Common Internet File System): Used for Windows systems. o Characteristics: Provides large-capacity storage with millisecond-level latency, suited for scenarios needing file sharing among multiple compute nodes. 3. Object Storage: o Characteristics: Manages data in a flat structure, which maintains high performance even with large numbers of objects. Supports REST APIs. o Advantages: High scalability and durability. o Challenges: Requires adaptation of traditional applications to work with object storage systems. Enterprise Storage Types 1. Direct Attached Storage (DAS): o Description: Connects an external storage device directly to a server via SCSI or FC interface, making it part of the server. o Characteristics: Data and operating system are not separated. 2. Network Attached Storage (NAS): o Description: Connects storage devices over a private network using TCP/IP, ATM, and FDDI, separating storage from file servers. o Characteristics: Integrates storage devices with network interfaces for shared file access. 3. Storage Area Network (SAN): o Description: A high-speed network connecting storage arrays to servers through switches. o Characteristics: Provides high performance and reliability for data storage and access. Distributed Storage Concept: Virtualizes storage resources across all hosts in an enterprise, storing data in multiple locations to improve reliability and access efficiency. Features: o Replication: Ensures data availability and consistency by maintaining multiple copies.  o Fault Tolerance: Automatically switches services from faulty nodes to others. o Scalability: Handles large-scale data growth efficiently. HUAWEI CLOUD Storage Services 1. Elastic Volume Service (EVS): o Description: Provides persistent block storage with high availability and low latency. Users can format the disk and create file systems for persistent data storage. o Use Case: Suitable for ECSs (Elastic Cloud Servers) and BMSs (Bare Metal Servers). 2. Scalable File Service (SFS): o Description: A NAS service offering scalable, high-performance file storage. Supports shared file access across various cloud resources. o Use Case: Ideal for scenarios needing shared access across ECSs, BMSs, and containers. 3. Object Storage Service (OBS): o Description: Provides scalable and secure cloud storage for unstructured data. Supports REST APIs for easy data access. o Use Case: Suitable for storing large amounts of unstructured data.