Load Balancing in Data Centers

Cloud Computing and Edge Computing

• Cloud computing is a network-based system where clients share configurable resources.
• Cloud-based systems use large data centers for services.
• Cloud computing has radically changed how computer services are built, managed, and delivered.
• Cloud computing involves numerous servers connected to a network (typically the internet).

Topic Outline

• Background: Cloud computing and data centers, data center networks (DCNs), network virtualization
• Example Cloud and Edge Computing implementation

Background

• Chapter 16: Cloud Computing and Network Virtualization

Cloud Computing and Data Centers

• Section 16.1: Cloud Computing and Data Centers
• A cloud-based system is a network-based computing system where clients use shared configurable resources.
• Cloud-based systems deliver services from large data centers.
• Cloud computing signifies various computing concepts.
• A large number of servers are connected to a network, usually the internet.

Driving Forces Behind Birth of Cloud Computing

• The possibility of creating overcapacity in large corporate data centers.
• Falling cost of storage.
• Ubiquity of broadband and wireless networking.
• Improvements in networking technologies.
• Cloud computing is equivalent to distributed computing over a network.
• A host server in a data center is called a blade.

Cloud Computing and Data Centers

• A blade consists of CPU, memory, and disk storage.
• Blades are stacked in racks; each rack averages 30 blades.
• Server racks are racks of host blades.
• Virtualization is crucial for cloud computing success.
• Cloud computing delivers computing services from a large virtualized data center to numerous users using shared applications.
• Services are provided through physical and virtual servers in data centers.
• Virtual servers are software-emulated servers that run on real servers.
• They can be moved and scaled without affecting the user.
• Cloud computing is on-demand, self-service, and metered.
• Cloud data centers provide various computing resources.
• Services include database storage, computing, email, voice, multimedia, and enterprise applications.

Benefits of Cloud Computing from Application Standpoint

• Benefits for infrastructure-less organizations and client users include flexibility, availability, use of platforms, commoditized services, and readily available APIs.

Types of Cloud

• Two main cloud types:
  • Public cloud: resources available dynamically to the public.
  • Private cloud: the on-site cloud of a corporation.
  • Hybrid cloud: a combination of on-site and off-site computing resources.
  • Community cloud: multiple organizations with common needs share a common infrastructure.

An Overview of a Cloud Computing System

• A data center consists of numerous server racks (server farm)
• A data center delivers various computing, storage, or other services depending on the application.
• Cloud storage is an important element of a data center.
• Server racks can contain a storage system for public or private data.
• A data center has its own DCN (data center network) to interconnect its host machines (servers) and connect it to the internet.
• A load balancer monitors incoming traffic and distributes it evenly across servers and storage systems.

Cloud Computing Service Models

• Cloud computing provides remotely accessible services.
• The advantage of a service model is that clients can rent software or hardware services remotely without upgrades.
• The most common service models in cloud computing are:
  • Software as a Service (SaaS): Software, including associated data, hosted centrally in the cloud (on-demand software).
  • Infrastructure as a Service (IaaS): Outsourcing the use of necessary equipment like storage, servers, and networking components.
  • Platform as a Service (PaaS): Provides a computing platform and a solution stack.

Data Centers – Design Principles

• Data centers are vital to individuals, enterprises, service providers, and content providers.
• Load balancing evenly distributes traffic within a data center.
• Effective routing protocols must guarantee high application performance.
• Energy consumption control is a significant design consideration.

Virtualization In Data Centers

• Creating a virtual version of a computing component such as operating systems, servers, or storage devices is called virtualization.
• Virtualization decouples infrastructure services from physical system components.
• Virtualization affects various aspects of daily life, including online shopping, education, and gaming.

Need For Virtualization Of Resources

• Sharing resources: When a resource (e.g., server) is too large, it can be virtually partitioned to share it.
• Aggregating resources: When a resource is too small, it can be virtually aggregated to create larger capacities.
  • Example: many inexpensive disks form a reliable, large storage system.
• Resource management: Virtualization makes managing devices and their networking, easier.
  • For example, users can be isolated when sharing a resource.

Virtualization in Data Centers

• A hypervisor is a software, hardware, or firmware that monitors and manages virtual machines.
• A host machine is the real computer (machine) on which a hypervisor is running.
• A guest machine is each individual virtual machine.
• A hypervisor manages guest operating systems, virtualizing hardware resources.

Benefits Of Machine Virtualization

• Effective Use of a Machine: Multiple users share computing and infrastructure facilities.
• Lower System Maintenance Cost: Machine maintenance cost can be reduced.
• Rapid and Dynamic Provisioning of new features: Feature agility through faster creation of VMs.
• Load balancing at and beyond data centers: Even distribution of computing load.

Challenges Of Machine Virtualization

• Virtualization poses networking challenges.
• Transport layer protocol performance can degrade.
• End-to-end packet delays can increase even with low network load.
• Latency of scheduling hypervisors.

Data Center Networks (DCNs)

• Typical data centers use routers and layer 2/3 switches in a specific topology.
• Servers in each rack are directly connected through top-of-rack (ToR) or edge switches.
• Each server in each rack has a network interface card (NIC) connecting to the ToR switch, and a data-center IP address.
• Edge switches are connected to multiple layer 2/3 aggregate switches.
• Aggregate switches are ultimately connected to layer 3 core switches.

Data Center Networks (DCNs)

• Data center networks (DCNs) use a hierarchical architecture.
• Scaling up DCNs to accommodate a larger number of hosts is possible because of this architecture.

Two types of traffic flow through DCNs:

• Traffic between clients outside the data center and data center hosts.
• Traffic between data center hosts.
• Boundary routers on top of core switches handle connections between data center servers and clients on the internet.

Load Balancer

• Load balancers are used in data centers to distribute traffic among servers to improve performance and availability.
• Load balancers ensure even distribution of requests from clients to servers.
• A large data center handles numerous concurrent real-time applications from diverse clients.
• Load balancers have many instances in large data centers to serve diverse applications.
• Load balancing can lead to congestion when many requests are handled and there are limited active machines.

Load Balancer - Round Robin Algorithm

• Load balancers use a round-robin algorithm to distribute requests sequentially to free, eligible servers.
• Load balancers don't collect and store the present state of the servers.

Load Balancer - Least-Connect Algorithm

• Load balancers employ the least-connect algorithm to distribute traffic based on active connections.
• Load balancers monitor active connections to servers.
• Load balancers prioritize servers with the fewest active connections, leading to more efficient load balancing in large-scale data centers.

DCN Architectures

• Switches and routers, load balancers, and data center links significantly contribute to data center infrastructure costs.
• Intelligent design of the networking section of cloud computing is crucial for reducing network costs.
• Three popular networking structures used in DCNs:
  • Server-routed networking scheme
  • Switch-routed networking scheme
  • Fully connected optical networking scheme

Network Virtualization

• Network virtualization is decoupling networking services from network infrastructures.
• It's crucial for shared network infrastructure in cloud computing.
• Network virtualization improves how networking and computing resources work and are optimized.
• Key challenges include general delay in computer networks and lack of seamless wireless connections.
• Network virtualization allows multiple independent, application-specific virtual networks to share a common physical infrastructure.
• This virtualization is achieved via software-defined networking (SDN).

Network Virtualization Components

• Virtual networks are decoupled from physical networks for maximum potential.
• Virtualization's software handles packet forwarding, decoupling from physical forwarding.
• The creation of virtual links, virtual switches, virtual NICs, virtual routers, and virtual load balancers from virtualization software.

Virtual Links

• Multiple hosts connected through a shared physical link.
• The link is viewed as a multichannel medium to transmit data through.
• Multichannel connectivity (channnelization) is referred to as a virtual link.

Virtual NICS

• A network interface card (NIC): attached to a host or network device.
• Processes Ethernet and Wi-Fi protocols.
• NICs on top-of-rack (ToR) switches or aggregate switches.
• Virtual NICs (VNICs) are necessary when virtual machines move between subnets in a data center.
• Any physical host or networking device requires at least one NIC.
• Every active virtual machine needs its own virtual NIC.

Virtual NICS

• IP addresses of virtual machines change while their MAC addresses remain unchanged.
• When virtual machines' network connections cross multiple layer 2 networks, a new IP address is required for the machine.

Network Interface Card(NIC) Virtualization

• A physical machine can host multiple virtual machines (VMs)
• Each VM has a virtual NIC (VNIC) and is housed in the physical machine
• A Virtual Ethernet Port Aggregator (VEPA) manages communication and switching among VNICS.

Virtual Switches (vSwitches)

• Virtual switches (vSwitches) are necessary when virtual machines needs to communicate with a greater number of hosts and need more ports than a single layer 2 switch can offer.
• vSwitches extend the number of ports on a physical switch, increasing flexibility when multiple virtual machines are active.
• Embedded virtualization software enables the addition of vSwitches to virtual or physical machines.

Edge Cloud Operations

• Edge cloud operations focus on operationalizing a cloud beyond core data centers.
• Edge clouds are often found in enterprises or Internet of Things (IoT) settings like factories.
• The edge places the location where cloud services connect to the physical world.

Edge Cloud Architecture

• Two subsystems: the edge cloud and the hybrid cloud.
• Aether is a platform as a service (PaaS).
• Aether uses open-source components.

Edge Cloud - Architecture

• Overview of Aether as a hybrid cloud featuring edge apps using the 5G data plane.
• Management and control-related workloads live in a central cloud.

Edge Cloud - ACE

• Aether edge cloud is ACE – Aether Connected Edge.
• A platform with Kubernetes – leaf/spine switching fabric.
• Manages the architecture using a software-defined fabric.

Hybrid Cloud

• Aether hybrid cloud has two subsystems in the central cloud:
  • Multiple instances of the Mobile Core Control Plane (CP).
  • Aether Management Platform (AMP)
• Pairing of CP (central) with UP (edge) instances is dynamic.

Summary

• Cloud computing and related concepts.
• Key aspects of network virtualization.
• The different types of clouds.
• Practical illustrations given in examples and relevant diagrams.