Distributed Systems and Kafka Quiz

Operating Systems

• Definition: Software managing hardware resources and computer programs.
• Key Functions: Memory management, process scheduling, and input/output operations.
• Types: Single-user, multi-user, real-time, and distributed.

Containerization

• Containers: Lightweight, portable units packaging applications and dependencies for consistent environments.
• Container Runtime: Software executing and managing containers (e.g., Docker).
• Container Images: Read-only templates used to create containers, containing application code and dependencies.

Virtualization

• Hypervisor: Layer allowing multiple virtual machines (VMs) on a single physical machine.
• Virtual Machines (VMs): Emulated computers running operating systems and applications as if they were physical machines.

Distributed Systems

• Definition: Components on networked computers communicate and coordinate through messages.
• Key Concepts: Scalability, reliability, and resource management are crucial.
• Microservices: An architectural style structuring applications as a collection of loosely coupled services.

Infrastructure

• Definition: Underlying physical and virtual resources supporting applications and services (servers, storage, networking).
• Management: Overseeing deployment, scaling, and maintenance of infrastructure components.

Automation and Scripting

• Importance: Automation tools and scripting languages streamline processes, manage configurations, and deploy applications.

Networking

• Protocols: Rules governing data communication over networks.
• Key Components: Protocols facilitating user interactions (e.g., HTTP, FTP, DNS), data transfer (e.g., TCP, UDP), network management (e.g., IP addressing & routing). Data Link layer components (e.g., MAC address).

TCP/IP Model

• A framework for understanding network communication.
• Consists of five layers: Application, Transport, Network, Data Link, & Physical.
• Each layer has specific functions.

Explanations of Difficult Terms

• Containerization: Method of virtualization allowing applications to run within isolated environments sharing the same operating system kernel.
• Hypervisor: Software creating and running virtual machines, sitting between hardware and OS, managing resource distribution.
• Orchestration : Automated arrangement, coordination, and management of complex computer systems, middleware, and services, commonly in container environments.
• Scalability: The capability of a system to handle an increasing amount of workload (e.g. increasing number of users to a website).
• Microservices: Architectural style structuring applications as small, independent services that communicate over a network.
• CLI: Text-based interface for interacting with software and operating systems.
• Docker: Platform for automating application deployments within containers.
• Asciinema: Tool for recording terminal sessions as text-based videos for documentation and tutorials.

Operating Systems and Linux

• Definition: Software managing computer hardware and software resources, providing a user interface and facilitating interaction between users and the computer.
• Types: Batch, time-sharing, distributed, real-time, and embedded.
• Kernel: The core part of the OS managing system resources and communication between hardware and software.
• Shell: User interface for interacting with the operating system through command-line input.
• Utilities: Tools and programs performing specific tasks to support user operations and system management.
• RAM: Volatile memory for storing data actively used or processed.
• Addressing: RAM locations identified by unique hexadecimal addresses.
• Address Register: Holds the address of the memory location to be accessed.
• Process: An instance of a program in execution.
• Job, Thread, Task: Units of work scheduled for execution in a system.
• Scheduling Algorithms: Techniques for managing process execution (e.g., FCFS).

Memory Management

• Techniques: Paging and segmentation for efficient data storage and retrieval.
• Virtual Memory: Allows execution of processes not entirely in memory.

Performance Metrics

• CPU utilization: Keeping the CPU constantly occupied and as busy as possible.
• Throughput: Number of processes completed in a given time.
• Turnaround Time: Total time taken to execute a specific process.
• Response Time: Time taken from submission to initial response.

Security and Permissions

• Mechanisms to protect against unauthorized access and ensure secure system operation.

Distributed Systems

• Overview of how resources are managed across multiple systems.
• Communication and coordination among distributed components.

Networking Protocols

• TCP/IP Model layers: Application, Transport, Internet, Data Link, Physical.
• Key Components: MAC address, Switches, and error detection/correction.
• Protocols: HTTP, FTP, and DNS (Application Layer), TCP/UDP (Transport Layer). IP and routing (Internet Layer), lower layer (physical and data link layer) protocols

Network Protocols

• WebRTC: Real-time audio, video, and data sharing between devices and browsers.
• Key Protocols: RTP (Real-time Transport Protocol), RTCP (RTP Control Protocol).
• Video on Demand: Protocols used for delivering video content (e.g., HLS, DASH, RTMP).
• Data Processing Methods: Batch processing and stream processing.
• Apache Kafka: Distributed event streaming platform for high-throughput data pipelines and streaming applications.
  • Components: producers, consumers, brokers, topics, partitions
  • Methods: replication, fault tolerance and consumer offsets for guaranteeing data reliability.

Kubernetes

• Kubernetes Overview: Automation of containerized application deployment, scaling, and management.
• Architecture: Control Plane (API server, scheduler, controller manager) and Worker Plane (nodes running applications in containers).
• Kubernetes Objects: Pods, deployments, services.
• Configuration Management: ConfigMaps, secrets.
• Automation and Scaling: Autoscaling, cronjobs.
• Health Checks: Mechanisms monitoring application health.
• Network: Networking options for managing traffic.
• Persistent Storage: Solutions for data persistent beyond the lifecycle of an application.
• Deployment Strategies: Techniques for rolling out new application versions (rollouts/rollbacks).

File Systems Overview

• Types of file systems (e.g., NTFS, FAT32, EXT4, APFS).
• Data Structures: Inodes, metadata.
• Storage Devices: HDDs, SSDs.
• Mounting/Unmounting.
• Performance and Optimization: Fragmentation, caching mechanisms.
• Backup and Recovery: Redundancy, self-healing.
• Emerging Technologies: NVMe, ZFS, Btrfs
• Directories and File Organization
• Sectors and Blocks

Cache and Storage Units

• Cache: Small, fast volatile memory storing frequently accessed data from main memory.
• Storage Units: Primary (e.g.RAM) and Secondary Storage (e.g. Hard Drives).
• Cache Hits and Misses
• Eviction Policies
• Volatile/Non-volatile memory

Security

• Confidentiality, Integrity, and Availability
• Detect Attacks and Intrusions
• Implement Safety Measures

Performance

• Control Resource Demand
• Limit Event Response
• Remove Computational Overhead
• Manage Resources

Energy Efficiency

• Monitor Resources, Metering
• Static Classification, Resource Allocation
• Usage Reduction, Event Management
• Event Prioritization

Modifiability

• Increased Cohesion (Module Consolidation)
• Modular Splitting
• Responsibility Redistribution
• Reduce Coupling (Component Independence)
• Encapsulation (Data Hiding)
• Intermediary Usage (Component Abstraction)
• Deferred Binding (Dynamic Connections)
• Component Replacement

Testing

• Control and observe system states.
• Specialised Interfaces.
• Limit Complexity
• Limit Structural Complexity

12-Factor App

• Codebase, Dependencies, Configuration, Backing Services
• Build/Release/Run, Processes, Port Binding, Concurrency

Digital Certificates

• Authentication and verification of public keys in secure communication networks.

Distributed Systems

• Fault Tolerance
• Data Consistency
• Scalability
• Raft Consensus Algorithm

Test your knowledge on the key concepts of distributed systems, particularly focusing on Kafka and its components. This quiz covers topics such as microservices architecture, hypervisors, and file systems, providing insights into how these elements interact within cloud environments.