Lecture 2.pdf

Week 2 | Lecture COMP 30520 Cloud Computing Lecturer: Dimitris Chatzopoulos Email: [email protected] O ce: E3.13 O’Brien Center for Science ffi  How was the practical yesterday? Part 1: - Setup your environment...

Week 2 | Lecture COMP 30520 Cloud Computing Lecturer: Dimitris Chatzopoulos Email: [email protected] O ce: E3.13 O’Brien Center for Science ffi  How was the practical yesterday? Part 1: - Setup your environment and test Docker Part 2: - Use an existing Docker image https://www.docker.com/ Virtualisation  Virtualisation Virtualisation: Is the illusion of creating or having two or more entities where the is only one physical entity in the system. In the context of (cloud) computing, virtualisation techniques can be used to make one compute node (i.e., one server) to appear as multiple. Similarly, a computer can run multiple operating systems simultaneously. Other use cases: Virtual Private Networks (VPNs) and Virtual Storage. Without virtualisation, apps developed for speci c hardware cannot run on other architectures. The hypervisor is a software program that enables hardware to host multiple Virtual Machines (VMs). Hypervisors allocate physical resources to VMs dynamically. app M app N app L app 1 app 2 app 3 app 4 app 5 app 6 VM i VM j VM k app 1 app 2 app N OS i OS j OS k Operating System (OS) Hypervisor Hardware Hardware fi  Virtualisation Bare-metal hypervisors run on the physical hardware of the host machine Hosted hypervisors run on on top of the existing OS A Virtual Machine Manager/ VM i VM j VM k Monitor (VMM) is a uni ed management and intuitive hypervisor software program that handles the orchestration of multiple VMs. You can install Hypervisor OS VMMs in multiple ways for di erent types of virtualisation Hardware techniques ff fi Server Virtualisation: (1) a VMM is deployed on the server, (2) the single physical server is divided into multiple virtual servers for resource sharing. Hardware Virtualisation: (1) a VMM is directly installed on the hardware system, (2) VM hypervisor manages the memory, processor and hardware related resources OS Virtualisation: (1) a VMM is installed on the OS, (2) useful for simulating multiple environments in parallel App App App App App a a b a c VM ja VM jb VM jc VM ka VM kb App App App App a b c VMM a VMM VM i VM j VM k VMM Physical Machine  Server Virtualisation: (1) a VMM is deployed on the server, (2) the single physical server is divided into multiple virtual servers for resource sharing. (+) E cient and reliable backup and recovery. (+) Supports IT operations automation and infrastructure scaling. (-) Signi cant upfront costs. (-) May not support proprietary business apps. (-) Lower security and data protection due to physical hardware sharing. Hardware Virtualisation: (1) a VMM is directly installed on the hardware system, (2) VM hypervisor manages the memory, processor and hardware related resources (+) Reduced maintenance overhead. (+) High delivery speed and rate of return with quality of information. (+) Minimal required changes in guest OS. (-) Requires explicit support in the host CPU. (-) Limits scalability and e ciency due to CPU overhead. (-) Risk of data damage as data are stored in the system OS Virtualisation: (1) a VMM is installed on the OS, (2) useful for simulating multiple environments in parallel (+) Multiple VMs operate independently and support di erent OSs. (+) Limited impact of malfunctions as crashes impact only VMs. (+) VM migration between servers (-) Signi cant admin overhead to maintain, update and secure host OS. (-) Heavy le system consumption due to le duplicates. (-) Heavy consumption of system resources. fi ffi fi fi fi ff ffi  Para-Virtualisation Para-virtualization is a type of virtualization where the guest operating system is modi ed to be aware of the hypervisor, allowing it to communicate directly with the hypervisor for certain operations. This awareness enables the guest OS to bypass some of the ine ciencies of full virtualization, where hardware is emulated, leading to improved performance. In a fully virtualized environment, the hypervisor needs to emulate all hardware interactions, which can slow down the performance, especially for operations like memory management and I/O. In para-virtualization, the guest OS is modi ed to replace certain instructions that would normally interact with hardware directly, allowing those instructions to call the hypervisor instead. These direct calls, make operations more e cient by avoiding the need for hardware emulation. By eliminating the need for certain hardware emulation, para-virtualization leads to better overall performance, especially for tasks involving CPU and memory. With para-virtualization, guest OSes can interact more e ciently with the hypervisor, reducing the processing and memory overhead required for virtualization. The system can better utilize hardware resources since the hypervisor doesn't need to spend time and resources emulating hardware that the guest OS interacts with. fi ffi ffi ffi fi  Hybrid Virtualisation Hybrid Virtualization is a combination of both full virtualization and para-virtualization, leveraging the strengths of each to provide both compatibility and performance optimizations. In hybrid virtualization, the system can run both fully virtualized guest operating systems (which don't require modi cations) and para-virtualized guest OSes (which are aware of the hypervisor and optimized for better performance). In hybrid virtualization, the hypervisor supports both modes: — Fully Virtualized Mode: For operating systems that do not support para- virtualization, the hypervisor uses traditional full virtualization techniques, such as hardware emulation or hardware-assisted virtualization (Intel VT-x, AMD-V). — Para-Virtualized Mode: For operating systems that support para-virtualization, the hypervisor allows the guest OS to directly communicate with it, bypassing hardware emulation and improving performance for CPU, memory, and I/O operations. Flexible virtualization approach that combines the advantages of full and para- virtualization. It allows a system to run unmodi ed guest operating systems in a fully virtualized environment while providing performance improvements for modi ed OSes that can interact directly with the hypervisor. This hybrid approach balances compatibility and e ciency, making it a popular choice in data centers and cloud environments that require support for a wide range of operating systems. ffi fi fi fi  VMs vs Containers In traditional virtualization, a hypervisor virtualizes physical hardware. The result is that each virtual machine contains a guest OS, a virtual copy of the hardware that the OS requires to run and an application and its associated libraries and dependencies. VMs with di erent operating systems can be run on the same physical server. For example, a VMware VM can run next to a Linux VM, which runs next to a Microsoft VM, etc Instead of virtualizing the underlying hardware, containers virtualize the operating system (typically Linux or Windows) so each individual container contains only the application and its libraries and dependencies. Containers are small, fast, and portable because, unlike a virtual machine, containers do not need to include a guest OS in every instance and can, instead, simply leverage the features and resources of the host OS. While there are still many reasons to use VMs, containers provide a level of exibility and portability that is perfect for the multicloud world. When developers create new applications, they might not know all of the places it will need to be deployed. Today, an organization might run the application on its private cloud, but tomorrow it might need to deploy it on a public cloud from a di erent provider. Containerizing applications provides teams the exibility they need to handle the many software environments of modern IT. ff ff fl fl  OS Server Hardware Aspect Virtualisation Virtualisation Virtualisation Virtualizing hardware Running isolated applications/ Creating virtual servers on a Focus resources for multiple environments on the same OS physical server VMs Operates at the Operates at the OS level Layer Operates at the server/OS level hardware/CPU level (sharing the OS kernel) CPU and hardware Technology Containerization Hypervisors extensions Example Docker, LXC, OpenVZ VMware, Hyper-V, KVM Intel VT-x, AMD-V Tech Containers share the same VMs have their own Each virtual machine has its Isolation OS kernel but are isolated virtual hardware, which own OS from each other maps to physical High: Lightweight and less Medium: VMs are more hardware Dependent on hardware Resource overhead (no full OS for each resource-intensive than but supports full E ciency container) containers isolation Common Running microservices, Data center consolidation, E cient VM Use cloud-native applications, running multiple VMs on one management at the Case DevOps, CI/CD pipelines server hardware level Dependent on the Startup Fast: Containers can start in Slower: VMs need time to boot hardware but generally time seconds their own OS slower than containers ffi ffi  OS vs Server Virtualisation In OS virtualization, containers share the host OS kernel, making them lightweight and e cient. In contrast, server virtualization uses hypervisors to run multiple operating systems (each with its own kernel) on a single physical server, which adds more overhead. OS vs Hardware Virtualisation OS virtualization does not virtualize hardware directly. Instead, it virtualizes at the OS level, allowing applications to run in isolated environments. Hardware virtualization, on the other hand, virtualizes the physical hardware (CPU, memory) to create independent virtual machines. Server vs Hardware Virtualisation Inserver virtualization is focused on creating multiple virtual servers on a single physical server, while hardware virtualization deals with the underlying technology that allows physical hardware to be abstracted and shared among virtual environments. Both work together in modern virtualization platforms. ffi  Other types of Virtualisation Type Focus Purpose Example Technology Virtualizing desktops for Enable access to virtual desktops, regardless MS Remote Desktop Desktop remote/local access of physical hardware Services Running applications in Simplify application deployment, avoid MS App-V, VMware Application isolated environments con icts, and improve compatibility ThinApp Abstracting physical Create virtual networks that operate Software-De ned Network network resources independently of the physical network Networking (SDN) Pooling and abstracting Simplify storage management and improve MS Storage Spaces Storage physical storage devices scalability Direct Abstracting data from its Provide uni ed access to data across multiple Red Hat Data Data physical location sources without moving or replicating data Virtualization Virtualizing GPU Enable sharing of GPUs across VMs for NVIDIA GRID, AMD GPU resources graphics-intensive workloads MxGPU, Intel GVT-g Abstracting physical Optimize memory usage and allocation Memory VMware ESXi memory resources among virtual machines or applications Virtualizing input/output Allow sharing of I/O devices among VMs, Single Root I/O I/O devices improving resource utilization Virtualization (SR-IOV) Virtualizing security Deploy scalable and exible security Cisco Firepower Virtual Security functions measures in virtual environments Firewall fl fi fi fl Containerisation  Containerisation Containerisation: Is an operating system virtualisation that builds and encapsulates software code, including dependencies, as a package that you deploy uniformly across any cloud infrastructure. Containerisation speeds up the application development process and makes it more secure by eliminating single points of failure. It also enables you to handle the problem of porting code e ectively from one infrastructure to another. It is easy to execute code independently on multiple clouds because the container package is independent of the host OS. Containerisation eliminates the problem of cross-infrastructure code management for building a code package with the application code and associated libraries required for code execution. Lightweight and E cient Portability Characteristics Improved Single of containers: Isolation Fault Isolation Executable Security Package Easy Operational Management ffi ff  Characteristics of containers Portability: Develop the app once and run multiple times Lightweight and Uses OS kernel and not the complete OS. E cient: Containers are smaller in size and require less start-up time Single Executable Allow packaging of application code including libraries and Package dependencies into one software bundle Isolation Execute in a dedicated process space. Multiple containers can run on single OS. Improved Reduce the risk of transmission of malicious Security code between containers and host invasion. Fault Isolation Minimal impact on adjacent containers if fault occurs in one speci c container. Easy Operational Allow automation of install, scale, and management Management of containerised workloads and services. ffi fi Components of containerised apps Container Host: The system software that executes containerised processes. It is a host running on VM or an instance in the cloud. Registry Server: A registry server is a le server that stores container repositories. Containers push and pull repositories from the registry server via the connection interface set-up with a domain name system (DNS) designation and port number. Container A container image is an executable package comprising application code, Image: runtime executables, libraries, and dependencies. Images when executed in the container engine become active containers. Container A container engine processes the container image as per the commands Engine/ de ned in user requests. These requests pull images from repositories Runtime: and execute them to launch containers. The engine has an embedded runtime component that provides functionality such as setting up security policies, rules, mount points, and metadata, including communication channels with the kernels needed to start containers. Container A container orchestrator supports development, QA, and production Orchestrator: environments for continuous testing. A container orchestrator schedules workloads dynamically, including the provision of standardised application de nition les. fi fi fi fi Components of containerised apps A namespace is a design followed to separate groups of Namespace: repositories. A namespace can be a username, group name, or a logical name that share container images. A kernel namespace is a design followed to provide Kernel containers with dedicated OS features, such as mount points, Namespace: network interfaces, process identi ers, and user identi ers. Tags support the mapping of the di erent versions of the latest Tag: or best container images in the repositories. Tags allow labelling of the images when the builder generates new repositories. A container repository that stores Repositories: di erent versions of container images. A graph driver maps stored images in Graph Driver: the repositories to a local storage. ff fi ff fi A container contains an application A unikernel is a machine image that packaged alongside its dependencies contains everything necessary for (external libraries, system tools, etc) application execution, including the operating system component. This Containers run on top of a container property makes unikernels completely self- engine, such as Docker, containerd, or su cient and able to run independently on CRI-O, which connects them with the top of a bare metal hypervisor. additional operating system components. The tools for packaging applications into unikernels, such as UniK, MirageOS, and Therefore a single host OS kernel Clive, combine the app's source code with powers multiple containers at the same dependencies, device drivers, and OS time libraries. The nal product is a lightweight operating system capable of performing a single function - running the packaged Container k Container i Container j application. APP i APP j APP k Kernel Kernel a Kernel b Kernel c Hypervisor Hypervisor Hardware Hardware ffi fi  Unikernels Unikernels are specialised, single-address-space machine images constructed by using library operating systems. Unikernels shrink the attack surface and resource footprint of cloud services. They are built by compiling high-level languages directly into specialised machine images that run directly on a hypervisor, such as Xen, or on bare metal. Since hypervisors power most public cloud computing infrastructure, this lets your services run more cheaply, more securely and with ner control than with a full software stack. Containers vs Unikernels: Key di erences Share the kernel of the host OS Include a small kernel in each deployment unit Simple to create from an image Requires advanced skills to create Designed to run multiple processes Designed to run a single process The OS handles resource allocation The hypervisor handles resource allocation ff fi  Containers vs Unikernels: How to choose Choose containers if: Want a well-documented, supported solution. Docker and Kubernetes brought containers into the mainstream, so it is easy to nd answers to most container-related issues. Run an app that features complex workloads. While unikernel performance surpasses containers in single-thread processing, multi-thread workloads are still processed faster using containers. Prefer simple deployments. Unikernels are challenging to assemble and require extensive knowledge of both the packaged application and the virtualization technology. On the other hand, using container platforms such as Docker requires almost no additional technical knowledge. Choose unikernels if: Maximize security. With their simple design, unikernels help increase security by reducing the attack surface. Minimize resource consumption. Unikernels eliminate layers of abstraction and create a lightweight, resource-friendly deployment platform. Achieve complete platform independency. While containers can also be called independent, they rely on the Linux kernel. It means that their performance su ers on other OSs, such as Windows and macOS, where an additional level of virtualization is necessary. fi ff  Comparison of Unikernels with other Virtualisation types Aspect Unikernels Other Virtualisation types Minimal OS, built speci cally for a single Full OS (in VMs) or shared OS kernel (in OS application containers) Resource Extremely low: includes only the essential VMs have higher overhead; containers are Overhead parts of the OS lightweight but not more than unikernels Startup Very fast: unikernels can boot in VMs take longer to boot; containers are also time milliseconds fast but need more time than unikernels VMs provide strong isolation; containers High: smaller attack surface due to fewer Security have a shared kernel, leading to some components security concerns Single-purpose, optimized for speci c VMs and containers can run multiple apps Flexibility applications and services Ideal for microservices, IoT, cloud-native VMs for full app stacks, containers for Use case apps, and edge computing microservices and scalable apps fi fi https://dl.acm.org/doi/10.1145/2541883.2541895 More on Cloud Computing Delivery models  Cloud computing refers to the deployment of multiple workloads in a scalable manner to serve on-demand system requirements and network resources. Building a centralized pool of resources (including the management layer) is essential to handle the infrastructure, applications, platforms, and data. To reduce human intervention, you need to construct an automation layer to dynamically manage the resource allocation within the pool. You can opt for di erent models of cloud computing based on the requirements to host various types of products. Apart from the primary cloud computing models, you can also opt for the Function- as-a-Service (FaaS) model, which focuses more on the function rather than the infrastructure to execute code based on events. FaaS is a serverless way to execute modular pieces of code. FaaS lets developers write and SaaS update a piece of code on the y, which can then be executed in response to an event, such as a user clicking on an element in a web application. PaaS This makes it easy to scale code and is a cost- e cient way to implement microservices. IaaS Cloud Physical Infrastructure ffi fl ff  Serverless computing Serverless computing is a method of providing backend services on an as-used basis. A serverless provider allows users to write and deploy code without the hassle of worrying about the underlying infrastructure. A company that gets backend services from a serverless vendor is charged based on their computation and do not have to reserve and pay for a xed amount of bandwidth or number of servers, as the service is auto-scaling. Most serverless providers o er database and storage services to their customers, and many also have FaaS platform Lower costs: serverless computing is cost-e ective, as traditional cloud providers of Advantages of Serverless computing: backend services often result in the user paying for unused space or idle CPU time. Simpli ed scalability: Developers using serverless architecture don’t have to worry about policies to scale up their code. The serverless vendor scaling on demand Simpli ed backend code: With FaaS, developers can create simple functions that independently perform a single purpose, like making an API call. Quicker turnaround: Serverless architecture can signi cantly cut time to market. Instead of needing a complicated deploy process to roll out bug xes and new features, developers can add and modify code on a piecemeal basis. https://www.cloud are.com/learning/serverless/glossary/function-as-a-service-faas/ fi fi fl ff ff fi fi fi  Microservices Imagine taking an application, chopping it up into pieces, and running it as a collection of smaller parts instead of one monolithic whole. That's basically what a microservices architecture is. Each piece of the application is called a microservice, and it performs one service only, runs independently of the other parts of the application, operates in its own environment, and stores its own data. From the user's perspective, an application built with microservices has a single interface and should work just the same as an application designed as one stack. However, behind the scenes each microservice has its own database and runs separately from the rest of the application. In addition, microservices within the same application can be written in di erent languages and use di erent libraries. User Interface User Interface Business Logic Microservice Microservice Microservice Microservice Microservice Microservice Application Database DB DB DB Monolithic Architecture Microservices Architecture https://www.cloud are.com/learning/serverless/glossary/serverless-microservice/ fl ff ff  Microservices Resilience: Because the application is divided up, one part of the application breaking or Advantages of Microservices: crashing does not necessarily a ect the rest of the application. Selective scalability: Instead of scaling the entire application, only the microservices that receive a great deal of usage can be scaled. Easier to add or remove features: Features can be rolled out or updated one at a time, instead of updating the entire application stack. Flexibility for developers: Microservices can be written in di erent languages and each have their own libraries. Microservices can be deployed in a variety of ways; they can be part of a serverless architecture, hosted in containers, developed using PaaS, or, theoretically, used to build a locally hosted application. However, the advantages of building an application out of microservices are perhaps most apparent when the application is hosted in the cloud, either using containers or in a serverless architecture. A microservice is typically larger and can do more than a function. A function is a relatively small bit of code that performs only one action in response to an event. Depending on how developers have divided up an application, a microservice may be equivalent to a function (meaning it performs only one action), or it may be made up of multiple functions. https://www.cloud are.com/learning/serverless/glossary/serverless-microservice/ fl ff ff  Container Orchestration  Orchestration Container Orchestration is the automated arrangement, coordination, and management of containers. User Interface User Interface Business Logic Microservice Microservice Microservice Microservice Microservice Microservice Application Database DB DB DB Monolithic Architecture Microservices Architecture  In the real world User Interface Microservice Microservice Microservice Microservice Microservice Microservice Microservice Microservice Microservice Microservice DB Microservice Microservice Microservice DB DB Microservice Caching queueing system  … containerised microservices are the foundation for cloud-native applications Cloud-native applications are a collection of small, independent, and loosely coupled services. Practically, cloud-native app development is a way to speed up how you build new applications, optimise existing ones, and connect them all. Cloud-native applications are speci cally designed to provide a consistent development and automated management experience across private, public, and hybrid clouds. The development of Cloud-native applications is focused on architecture modularity, loose coupling, and the independence of its services. Each microservice implements a business capability, runs in its own process, and communicates via application programming interfaces (APIs) or messaging. This communication can be managed through a service mesh layer. A service mesh is a way to control how di erent parts of an application share data with one another. A service mesh is a dedicated infrastructure layer built right into an app. This visible infrastructure layer can document how well (or not) di erent parts of an app interact, so it becomes easier to optimise communication and avoid downtime as an app grows. fi ff ff  Challenges 1) Service discovery 2) Load balancing 3) Secrets con guration/storage/management 4) Health checks 5) Autoscaling, autorestart , etc of containers 6) Zero-downtime deployments Container orchestration platforms become extremely useful because they o er solutions to these challenges Kubernetes is an open-source platform for automating deployments, scaling, and operations of application containers across clusters of hosts, providing container- centric infrastructure. fi ff  Kubernetes Core Concepts Pod: The smallest and simplest unit in the Kubernetes object model that you can create or deploy. It represents a running process in the cluster. Can contain one or multiple containers. Worker node: Runs the Kubernetes agent that is responsible for running Pod containers via Docker, requests secrets or con gurations, mounts required Pod volumes, does health checks and reports the status of Pods and the node to the rest of the system. Service: An abstraction which de nes a logical set of Pods and a policy by which to access them. Generally it’s used to expose Pods to other services within the cluster or externally. Master node: Runs multiple controllers that are responsible for the health of the cluster, replication, scheduling, endpoints (linking Services and Pods), Kubernetes API, interacting with the underlying cloud providers etc. Generally it makes sure everything is running as it should be and looks after worker nodes. Deployment: Provides declarative updates for Pods (like the template for the Pods), for example the Docker image(s) to use, environment variables, how many Pod replicas to run, labels, node selectors, volumes etc DeamonSet: It’s like a Deployment but instead runs a copy of a Pod (or multiple) on all (or some) nodes. Useful for things like log collection daemons, node monitoring daemons and cluster storage daemons. fi fi  Kubernetes Core Concepts Cluster: is a collection of compute, storage, and networking resources that Kubernetes uses to run the various workloads that comprise your system. Note that your entire system may consist of multiple clusters. Labels: are key-value pairs that are used to group together sets of objects, very often pods. Label selectors are used to select objects based on their labels. Replication controllers and replica sets both manage a group of pods identi ed by a label selector and ensure that a certain number is always up and running. The main di erence between them is that replication controllers test for membership by name equality and replica sets can use set-based selection. Volume: Local storage on the pod is ephemeral and goes away with the pod. If the design of your system dictates data exchange between containers of one node, local storage is su cient. Sometimes it's important for the data to outlive the pod, or it's necessary to share data between pods. The volume concept supports that need. Secrets: are small objects that contain sensitive information, such as credentials and tokens. Names: Each object in Kubernetes is identi ed by a UID and a name. The name is used to refer to the object in API calls. ff ffi fi fi This book (and many others) is available online at UCD library (OneSearch) !!! https://www.ucd.ie/library/  Discussion: Cloud Economics in the era of FaaS and Microservices