COTS, Hardware, and Cloud Deployment Models

Product Deployment Options

• Software can be delivered online through the Software-as-a-Service (SaaS) model, which is not known by the customer.

Common-Off-The-Shelf (COTS) Hardware

• COTS hardware is standardized and bought in large quantities, reducing hardware variety in a data center.

Kubernetes "ClusterIP" Service

• A "ClusterIP" service is used for services reachable only inside the cluster, such as a backend database consumed by a web server frontend.

Kubernetes Ingress Controller

• An Ingress controller in Kubernetes enables multiple HTTP/HTTPS services to be delivered on a single TCP port through service multiplexing.

Kubernetes Service for Internal Reachability

• A Kubernetes "ClusterIP" service is used to create a service only reachable from inside a Kubernetes cluster.

Purpose of Kubernetes Service

• A Kubernetes Service (e.g., Cluster IP) handles cases with multiple replicas and pod respawning, unlike direct pod IP addressing.

Memory Requirements

• Memory requirements theoretically compare as: Memory(VM) > Memory(Docker) = Memory(bare hardware).

Disk Requirements

• Disk requirements theoretically compare as: Disk(VM) = Disk(Docker) > Disk(bare hardware), assuming no layered file system advantages.

Scalable Web Server

• Kubernetes on a public cloud is the best choice for creating a web server that scales endlessly.

Service Mesh Functionality

• Load balancing incoming HTTP/HTTPS connections to different service instances based on application-level information (e.g., requested URL) requires a "service mesh" software instead of a Kubernetes "service".

Real-Time Video Conferencing

• Buying a managed service (e.g., Zoom) running in multiple cloud locations across the world is the best option for setting up a real-time video conferencing service.

Docker Filesystem Modification

• A file is written in the file system of the running Docker when a Docker running in a server tries to modify its filesystem.

Docker Layered Filesystem

• The layered file system in Docker builds a new layer each time the Dockerfile executes a command that modifies the current filesystem.

Docker Resource Availability

• A Docker reports 104 CPU cores and 512 GB RAM if the server has 2 CPUs (52 cores each) and 512 GB RAM.

Purpose of Dockerfile

• The Dockerfile defines a "recipe" listing the steps to re-create the container from scratch.

Kubernetes "daemonset"

• A “daemonset” in Kubernetes is used for a pod that must always be running on each Kubernetes node to provide network services.

Kubernetes Cluster with "ClusterIP" Service

• An e-commerce site in a Kubernetes cluster with the web frontend configured as a "ClusterIP" service may be reachable from the Internet depending upon the configuration of the Ingress controller.

Kubernetes "Control Loop"

• The “control loop” concept in Kubernetes is implemented by controllers, which watch the state of the resources in the cluster, then make or request changes where needed.

Kubernetes Custom Resource

• A Custom Resource in Kubernetes enables the definition of generic objects, even not related to any Kubernetes workflow.

VM/Docker Network Connectivity

• By connecting a set of VMs/Dockers to a software bridge, you are NOT able to guarantee full network connectivity to the services running within the VMs/Dockers.

Data Center Servers

• Servers in a new data center are usually all equal in terms of CPU, memory, disks, etc.

High-Availability Database-as-a-Service

• For high-availability, an e-commerce site that leverages a database-as-a-service from the cloud provider should be set up in multiple replicas across different Regions.

Application Deployment

• An application deployment on a cluster in public cloud requires care about reliability, which is not completely guaranteed by the cloud provider.

Edge Computing

• The deployment of cloud computing technologies at the edge of the network is useful with applications that cannot afford network problems, require very low latency, and need to keep some data local, or that need to reduce the data volume before sending it to the cloud.

Computing Virtualization Technologies

• Technologies for computing virtualization are used whenever strong computing isolation is required.

Docker Filesystem

• The filesystem in a Docker at startup is completely independent from the one present on the hosting machine; however, temporary files created during Docker execution are stored in the filesystem of the hosting machine (e.g., /var/tmp/).

Docker CPU Cores

• A Docker running on a server that has 2 CPUs (52 cores each) will create 104 threads.

VM Reachability in OpenStack

• The reachability of a VM running in OpenStack from a customer connected to the Internet must be explicitly enabled in the OpenStack configuration of the VM.

TCP Connection

• When a client (on the Internet) establishes a TCP connection toward a pod/service: Two cascading TCP connections must be established to allow the client to connect to the pod.

Micro-service in a service mesh

• In a service mesh, each micro-service is automatically started with a sidecar that implements the service mesh.

Trap-And-Emulate Approach

• In computing virtualization, the Trap-And-Emulate approach can be used to emulate all the instructions that generate a trap.

CPU Para-virtualization

• Replaces some source code of the GuestOS kernel with the proper equivalent function in the HostOS kernel.

Modern CPUs

• Have many CPU rings, and 2 of them are intended for executing OS kernels.

Dynamic Binary Translation

• Can refer to an instruction executed by GuestOS kernel.

VMEntry and VMExit Instructions

• Enable fast transitions between GuestOS and HostOS kernels.

Computing Virtualization Hypervisor

• Is usually able to emulate a very small number of physical devices.

Operating System Image in a VM

• Usually includes drivers for many physical devices (e.g., Intel NIC), and a few para-virtualized devices (eg., VIRTIO).

Libvirt

• Simplifies the interaction with VMs running on different hypervisors (KVM, Vmware, etc).

Linux

• QEMU runs in user space, KVM runs in kernel space.

Hot Migration

• Implementing a hot migration of a VM to another server requires moving the state of The entire VM (i.e., applications and GuestOS kernel), plus some additional data kept in the hypervisor.

Sensitive Instruction in Computing Virtualization

• Is an instruction that may leak information about the current running kernel context

CPU ring 0

• Can generate a trap if it is a privileged instruction.

VMX root/non-root modes

• Is more efficient when handling system calls.

Userland Program Trap-And-Emulate

• We experience two transitions: User space --> HostOS kernel --> GuestOS kernel.

Direct Assignment for I/O Devices

• A real device that is completely controlled by the GuestOS.

GuestOS in a Virtualized Environment

• It usually detects a set of virtual devices presented to it by the hypervisor.

A VIRTIO Driver

• Does not use VMENTRY/VMEXIT operations.

KVM Hypervisor

• QEMU runs at ring 3 (VMX root), the GuestOS runs at ring 0 (VMX non-root).

Linux Tools virsh and virt-manager

• Enable users to interact with KVM-based VMs in a user-friendly way.

The state of a VM

• The content of the entire memory allocated to the VM plus some additional information stored in the hypervisor.

Computing Virtualization, Trap-And-Emulate approach

• Is used to emulate privileged CPU instructions executed at the wrong privilege level.

Instructions Executed on the CPU

• Will generate a trap if it is either a sensitive or privileged instruction.

VIRTIO

• Uses mainly bi-directional queues shared with the companion backend driver to send/receive data.

Dynamic binary translation

• Often translates a binary instruction executed by the GuestOS kernel.

Docker Network

• Provides a default (simplified) network behavior, but it can also offer high flexibility and customizations such as in a traditional virtualized environment (e.g., KVM).

Docker Network Connection

• Must use the IP address of the server, and the public TCP/UDP port published by Docker2.

Standard Docker Network

• It includes a software bridge, plus routing and NAT functions operating on all the traffic generated by the containers and directed outside of the server.

Docker IP address

• It is "manually" assigned by the Docker framework to the running container.

Network Connection

• It must use the IP address of Docker2, and the actual TCP/UDP port used by the application running in Docker2.