OSC Lecture 1 PDF

OSC Lecture 1 Chapter Summary: Introduction to Operating System Concepts Understanding Computer Functionality Analogy with Family: Comparing computer hardware to a family dynamic: Dad represents the computer’s CPU, handling requests but limited to one...

OSC Lecture 1 Chapter Summary: Introduction to Operating System Concepts Understanding Computer Functionality Analogy with Family: Comparing computer hardware to a family dynamic: Dad represents the computer’s CPU, handling requests but limited to one task at a time. Mom symbolizes the operating system, which prioritizes tasks and ensures the urgent needs are met first. This comparison helps illustrate how an OS handles multiple requests from various programs, akin to a parent managing the needs of different children. Why Learn About Operating Systems? Frameworks and Programming Languages: Understanding operating systems provides insights into various software tools and programming languages, such as: Java, Python, PHP, C, and C++, each with unique characteristics. The distinction between compiled (e.g., C++) and interpreted languages (e.g., Python), emphasizing different execution speeds and memory usage. Machine Code and Compilation: A compiler translates an entire program into machine code, which may take longer to analyze but runs faster. (more memory) OSC 1  An interpreter translates code one statement at a time, resulting in slower execution but greater memory efficiency. Operating System Characteristics Definition of Operating System: An OS is a body of software managing resources, allowing multiple programs to run seemingly simultaneously, managing memory, and facilitating communication between programs and devices. The Kernel: The Heart of the Operating System The kernel operates in kernel space (privileged mode) to manage system resources and processes, while user space (ordinary mode) restricts access to critical system functions. System calls and interrupts facilitate communication between user processes and the kernel, ensuring smooth operation and resource allocation. Historical Context and Evolution of Computers The evolution of computers began with early devices like the Chinese abacus and progressed to complex programmable machines like ENIAC. OSC 2  Ada Lovelace is recognized as the first computer programmer, having contributed to Charles Babbage’s Analytical Engine. Moore’s Law: Predicts the doubling of transistors on microchips approximately every two years, impacting computational power and efficiency. The Unix Philosophy and its Impact Core Principles: The Unix philosophy promotes building simple, modular programs that perform specific tasks well and can be easily integrated with others. This approach has influenced many modern operating systems, encouraging clear and maintainable code structures. System calls – software wants to execute I/O Traps – software executes an illegal instruction Hardware interrupts Software interrupts interrups = changes state) Kernelspace = system mode – priviliged mode – supervisor mode – secure mode – unrestricted mode Userspace = ordinary mode – user mode – restricted mode Lec 1 Q&A General Concepts 1. What does it truly mean if people say: “computers work on 0’s and 1’s”? Computers operate using the binary number system, where all data is represented as sequences of 0s (off) and 1s (on). This binary encoding forms the basis for all computations and data storage. 2. What is a “framework”? A framework is a structured platform for building applications, providing tools and libraries that streamline the development process. Examples include Angular for web apps and Django for Python. OSC 3  3. What is a “library”? A library is a collection of pre-written code that developers can use to perform common tasks, such as data manipulation or graphical rendering, without writing code from scratch. Examples include jQuery (JavaScript) and NumPy (Python). 4. What is the difference between a compiler and an interpreter? A compiler translates an entire program into machine code before execution, creating an executable file (e.g., C). An interpreter translates code line-by-line during execution, allowing for immediate feedback (e.g., Python). 5. What type of programming languages exist? Why are some called high- and others low-level programming languages? Programming languages include high-level (e.g., Python, Java) and low- level (e.g., Assembly, C). High-level languages are user-friendly and abstract hardware details, while low-level languages are closer to machine code and offer greater control over hardware. 6. What is the difference between assembly code and machine code? What is disassembly in this context? Assembly code uses mnemonics for machine instructions, making it human-readable (e.g., MOV AX, 1 ). Machine code is binary that the CPU executes directly. Disassembly converts machine code back to assembly for analysis. 7. What is a PCB (in a hardware context) and how are electronics and logical gates related? A PCB (Printed Circuit Board) connects and supports electronic components using conductive pathways. Logical gates (AND, OR, NOT) are implemented on PCBs to perform basic operations in digital circuits. 8. Give 5 names of different Operating Systems (flavors). Windows macOS OSC 4  Linux Android UNIX 9. Which families are still common today / most used? How old are they? Common families: Windows (1985), Linux (1991), macOS (2001), Android (2008). UNIX dates back to the 1960s and has modern derivatives. 10. What is a Linux distribution? A Linux distribution (distro) is a version of the Linux OS that includes the kernel, libraries, software, and package management, tailored for various user needs (e.g., Ubuntu, Fedora). 11. Give a definition for “Operating System” (in a classic sense and as defined in this course). Classic: An OS manages hardware resources and provides services for applications. Course context: An OS facilitates user interaction with hardware and manages application execution. 12. Do a microwave, a fridge or a classic Game Boy have operating systems? No, traditional microwaves, fridges, and classic Game Boys do not have full operating systems. They use embedded systems or firmware for specific tasks without the complexity of a full OS. 13. Explain the difference between virtualization, emulation, and simulation. Virtualization allows multiple OS instances on one hardware platform (e.g., VMware). Emulation replicates the behavior of one system on another OSC 5  (e.g., console emulators). Simulation models a system’s behavior for analysis (e.g., flight simulators). 14. What is a mainframe? A mainframe is a large, powerful computer designed for bulk data processing, capable of supporting multiple users and applications simultaneously, often used in enterprise and governmental operations. 15. What is/are the primary goal(s) of an Operating System? Primary goals: manage hardware resources, provide user interface, ensure application execution efficiency, maintain stability and security, facilitate inter-process communication. 16. Why is there a new way of looking at “Operating Systems” (in a cloud/microservice/more modern sense)? Modern OS concepts focus on cloud computing and microservices, emphasizing scalability, flexibility, and resource efficiency, moving away from monolithic systems to distributed architectures. 17. Explain the difference between User space and Kernel space. User space is where user applications run with restricted access to resources. Kernel space is where the OS core executes with full access to hardware, ensuring system stability and security. 18. What are system calls? System calls are APIs that allow user applications to request services from the OS kernel, enabling interactions with hardware, file management, or process control (e.g., open() , read() ). 19. What are drivers? Drivers are specialized software that allow the operating system to communicate with hardware devices, translating OS commands into device-specific actions (e.g., printer drivers). General Computer History (up to present day) 1. What made Pascal so special? OSC 6  Pascal was one of the first high-level languages emphasizing structured programming, portability, and ease of learning, influencing many subsequent programming languages and education. 2. Why are people like Charles Babbage and Ada Lovelace so critical in the evolution of (modern) computers? Babbage designed early mechanical computers (Analytical Engine); Lovelace is considered the first programmer for her work on algorithms, laying foundational concepts for modern computing. 3. Was there ever such a thing as a non-electronic computer? Yes, non-electronic computers include mechanical calculators and analog computers, which used gears and levers for calculations, like the abacus and Babbage’s Difference Engine. 4. How did Alan Turing win the war? Turing developed techniques to break the Enigma code used by the Germans, significantly aiding Allied forces by intercepting and decoding enemy communications. 5. Which functions does a punch card have for a computer? Punch card = set of instructions Punch cards input data and instructions into early computers through holes representing binary data, essential for programming and data processing before modern input methods. 6. What do we mean when we talk about the Von Neumann bottleneck? The Von Neumann bottleneck refers to the limitation in processing speed caused by the separation of memory and CPU in the Von Neumann architecture, affecting data transfer rates. 7. How does Moore’s Law impact your cost of digital living? Moore’s Law states that the number of transistors on a chip doubles roughly every two years, leading to increased performance and decreased costs, making digital devices more affordable. OSC 7  A Matter of Unix 1. Why was Unix made at all? Unix was created to provide a multi-tasking, multi-user OS that was efficient, portable, and suitable for research and development, emphasizing simplicity and functionality. 2. Why was Dennis Ritchie a crucial element in the evolution of the computer? Ritchie developed the C programming language and co-created Unix, providing the tools and environment that shaped modern software development and operating systems. 3. Who was Ken Thompson? Ken Thompson is a co-creator of Unix and the B programming language, instrumental in developing early computing systems and concepts. 4. What was the role of the evolution of B & C in the creation of Unix and subsequent Operating Systems? B laid the groundwork for C, which became the primary language for Unix and other OS, enabling efficient and portable software development. 5. What are the core principles of the Unix Philosophy? Core principles: Write programs to do one thing well, use text as a universal interface, build simple, composable tools for complex tasks. 6. Why is worse (sometimes) better in modern operating systems? “Worse is better” suggests simpler, less polished solutions may be more successful due to ease of implementation and adaptability over complex designs. 7. Sketch the three main family lines that evolved from Unix. BSD (Berkeley Software Distribution): Leads to FreeBSD, OpenBSD, NetBSD. System V: Influences commercial Unix versions like AIX, HP-UX. Linux: Created by Linus Torvalds, with many distros like Debian, Red Hat. 8. What is POSIX? OSC 8  POSIX (Portable Operating System Interface) is a set of standards ensuring compatibility between Unix-like OS, promoting portability of software across different systems. 9. What is UTF 32-16-8? UTF-32, UTF-16, and UTF-8 are Unicode encoding schemes: UTF-32: 4 bytes per character. UTF-16: 2 bytes for most; 4 bytes for less common. UTF-8: 1 to 4 bytes, space-efficient and ASCII-compatible. 10. What’s GNU? GNU (GNU’s Not Unix) is a free software project started by Richard Stallman in 1983 to create a complete Unix-like OS, emphasizing free software rights through the GNU GPL. 11. What is a distro? A “distro” is a specific version of Linux that combines the kernel with software applications and tools, tailored for different use cases (e.g., Ubuntu, Fedora). 12. How do you find out which distros are most used and why would you want to know? Check sites like DistroWatch for rankings based on user interest. Knowing popular distros helps choose one with a larger community, better support, and compatibility, enhancing usability. OSC 9 OSC 10  Exam Questions Question 1: Which one of the following statements is TRUE? 1. Unix came into being in the 1980s. 2. Ken Thompson & Dennis Ritchie made the C language because the B language wasn’t good enough to create "their" Unix OS. (TRUE) Explanation: The B programming language, created by Ken Thompson, was the precursor to C. Dennis Ritchie improved upon it to develop the C language, which was then used to write Unix. 3. Dennis Ritchie said UNIX is the spiritual successor of BSD. 4. UTF is a programming language that was used at Bell Labs. Question 2: Ubuntu is POSIX compliant (not certified, because that is EXPENSIVE). Which means you'll find which of the below in Ubuntu? 1. A command shell. 2. Support of regular expressions. 3. Sockets. 4. All of the other three. (TRUE) Explanation: POSIX compliance ensures the presence of essential features like a shell, regular expressions, and socket support, all of which are found in Ubuntu, even if it is not certified. Question 3: "Everything is a file" is a statement you hear a lot in the Unix/Linux world. This means that: 1. Your file system must always contain at least one file. 2. You can read from/write to a physical device by reading from/writing to a corresponding file in the /dev directory. (TRUE) Explanation: Unix/Linux treats devices, processes, and other system objects as files. This abstraction allows interaction with hardware and OSC 11  processes through file I/O mechanisms. 3. The entire userspace is contained in one single file. 4. The entire kernel space is contained in one single file. Question 4: Fill in the blank: You can find the configuration for mounts that will survive a system reboot in the file /etc/____. Correct Answer: fstab (TRUE) Explanation: The /etc/fstab file lists filesystems and their mount options. This ensures that specified filesystems are mounted automatically when the system reboots. Lecture 2 OSC 12 OSC 13  Docker docker is a tool(set) to manage containers, docker is installed on the OS an whatever bins/libs would be in a container then Rocket comparable to docker, but doesnt use deamon (lightweight vm) OSC 14  Dockerfile read from top to bottom Use Dockerfile to do anything you want that you could do in the command line last line would be CMD python (how you want to run it ) Containers are isolated processes Containerization = Operating -system-levvel virtualization OS feature where the kernel allows the existence of multiple isolated user- space instances “CPU isolation”: sequential execution of processes “memory isolation”: Standard process isolation: each process has its virtual memory space OS manages mapping to physical memory (and offloads it to disk) “Privilege isolation”: process run as administrator has different access to other processes and files (and resources) than regular user Docker hub repository Image names are [team or person name]/[project name]/[subproject]. mcr.microsoft.com/windows/nanoserver Or just [project name] for ‘official’ docker images hello-world OSC 15  nginx A tag is a label that helps identify a unique image (optionally). If tag name is not specified, Docker uses the latest tag. mcr.microsoft.com/windows/nanoserver:1809 nginx:alpine Linux containers LXC LinuX Containers covers just about every containment feature supported by the upstream kernel Focus on ‘system containers’, instead of ‘app container’ “lightweight VM” VMs without overhead of a separate kernel and simulating all hardware LXD Extension to LXC user interface to manage containers containers can be managed over the network through a REST API Commercial support via Canonical (i.e. the Ubuntu guys/girls) Using LXCFS A container aware Filesystem in userspace (FUSE), cfr. e.g. sshfs Docker in Windows Docker Uses Hyper-V in the background in Windows OSC 16  Lecture 2 Q&A OSC 17  Lecture 3 Recap of Lecture 2 Performance Topology: The closer a process is to the operating system (OS), the higher its performance potential. For instance, running multiple teams in a browser within a virtual machine (VM) on a Windows 10 host can introduce layers that affect performance. Virtualization and Cloud Computing: OSC 18  Virtualization refers to the creation of virtual (rather than actual) versions of physical computing resources. A hypervisor manages these virtual environments, with distinctions between Type I (bare-metal) and Type II (hosted) hypervisors. “As-a-service” terminology emphasizes cloud-based services, contrasting with “on-prem” solutions. Containers: Containers, exemplified by Docker, encapsulate applications and their dependencies, differing from traditional virtual machines. Docker employs a client-server architecture using REST API calls, simplifying software distribution. Processes: The Building Blocks of Computing Definitions: Job: An older term still referenced in literature, particularly in job scheduling contexts. Process: An active job with a state that includes program code and execution context, unlike a passive program or binary, which has no state. Process States: A process transitions through various states: New: Creation phase. Running: Actively executing instructions. Waiting: Awaiting an event. Ready: Awaiting processor assignment. Terminated: Completed execution. OSC 19  Process Control Block (PCB): The PCB is a data structure that stores essential information about a process, such as: Process state. Program counter (next instruction to execute). CPU registers. Memory management details. I/O status and open files. The PCB serves as a “state saver,” enabling multitasking by allowing a single CPU to manage multiple processes. Context Switching Definition: A context switch (saving/storing the state of process or a thread) occurs when the CPU transitions from one process to another, involving state saving and restoring. Factors influencing context switch actions include: Type of operating system. Presence of multiple register sets. Transitioning between user mode and kernel mode. OSC 20  Context switches enable multitasking, allowing several processes to run efficiently on a single CPU. Memory Management: Stack vs. Heap Stack: A linear data structure that operates on a Last In, First Out (LIFO) principle, often used for function calls and local variables. Faster to get something from the stack than the heap ⇒ on stack small data like numbers, string, bolean, pointers ,references to heap Heap: A more flexible memory area used for dynamic memory allocation. We can reserve memory for the process. This you need to programm yourself. ⇒ big data like a whole class instance (class) but they always have a reference to them in the stack as well ⇒ new = reserve bites in heap Recursion vs. Iteration: Recursion uses the stack for function calls, while iteration typically uses loops. Kernel Space vs. User Space OSC 21  Understanding the distinction between kernel space (where the OS executes) and user space (where user applications run) is crucial. Errors in the kernel can lead to system crashes, highlighting the importance of stability and error handling in OS design. Driver (device driver) = a way for the OS to connect with the elements of hardware itself ( run in kernel space) C Programming and Compilation The C programming language, developed in the early 1970s, is noted for its low-level capabilities. Compiling a C program can be done using commands like gcc hello.c -o hello or through a Makefile, which simplifies the build process. ldd command shared dependencies dmesg command = kernel messages Algorithms and Performance Algorithmic Complexity: The efficiency of algorithms is measured using Big- O notation, which indicates how execution time or memory consumption grows relative to input size. Examples include: Linear Search (O(n)): Check all elements one by one. Selection Sort (O(n²)): Requires multiple passes, making it less efficient for larger datasets. N = number of iterations, elements, processes OSC 22  O N^3 OSC 23  O(N) OSC 24  O(n)+O(n)=O(n) A while loop is O(n)O(n)O(n) if: 1. The loop condition depends on a variable that changes in a way that ensures the loop executes a linear number of times relative to the input size n. nn 2. The loop body performs work proportional to a constant amount for each iteration. Lecture 4 Thread: It’s inside process. A basic unit of CPU utilization that shares resources with other threads of the same process. EACH THREAD HAS A SEPARATE STACK, COUNTER, REGISTER. Shared : data, file ,memory OSC 25  Lightweight Process: Refers to the minimized overhead associated with thread management compared to processes. Blocking Thread: A thread that does not block other threads from executing, enhancing concurrency. Pthread_create: A system call to create threads within the same process. Threads vs. Processes The distinction between threads and processes is pivotal in understanding OS functionality. When a fork system call is executed, it creates a new child process, which can lead to complexities such as orphaned processes if the parent terminates before the child. In contrast, threads created using Pthread_create remain within the same process, avoiding the need for extensive memory clean-up. Fork vs. Pthread_create: Fork: Creates a new child process; child can become orphaned if parent exits. Pthread_create: Threads share the same memory space, leading to efficient resource use. in java when running a simple hello world there are a lot of threads because java runs in its own “virtual machine” OSC 26  8 times printed hello fork to the power of 3 2^3 Thread Scheduling Responsibility While process scheduling is primarily the responsibility of the OS, thread scheduling often falls on the application or program designer. This shift allows for more granular control over how threads operate and interact, ultimately impacting performance. OS Role: Manages CPU scheduling for processes. Application Role: Designs and implements thread scheduling mechanisms. OSC 27  remember different possibilities Importance of Threads in Multithreading Threads provide significant advantages, including improved responsiveness and efficient resource sharing. Multithreading allows applications to perform multiple tasks simultaneously, leading to enhanced performance. Advantages: Responsiveness: Applications remain responsive to user actions. Resource Sharing: Threads within the same process can share data and resources efficiently. Cost Efficiency: Managing threads is less resource-intensive than managing processes. Real-World Applications and Examples Consider a scenario in online gaming, such as League of Legends, where a server needs to handle multiple client connections. Using threads allows the server to create a thread for each client, promoting effective resource management and responsiveness. OSC 28  Example: In the case of a server handling multiple clients, threads allow for parallel processing without blocking, ensuring a smooth gaming experience. Concurrency Challenges and Solutions Despite the advantages, threading introduces challenges, particularly regarding concurrency and resource access. Issues such as race conditions can arise when multiple threads access shared resources simultaneously. The chapter discusses tools to mitigate these problems, such as mutexes and semaphores. Concurrency Issues: Race Conditions: Occur when multiple threads access shared data, leading to unpredictable results. Mutex: A locking mechanism that ensures only one thread can access a resource at a time. Semaphore: A signaling mechanism used to control access to shared resources. Key Takeaways: Threads significantly enhance application responsiveness and resource efficiency. Proper management of concurrency issues is essential for maintaining data integrity. Understanding the balance between threads and processes is critical for effective OS design and application performance. Lecture 4 Q&A 1. What are the differences between processes and threads? Processes are independent; threads are lightweight. Processes have separate memory; threads share memory space. Example: Web browser (process) with multiple tabs (threads). 1. What are the similarities between processes and threads? OSC 29  Both can run concurrently(at the same time). Both have unique identifiers (PID for processes, TID for threads). Both can communicate with each other. 2. What are the benefits of threads? Improved application responsiveness. Efficient resource sharing. Lower overhead compared to processes. Example: A server handling multiple client requests simultaneously. 3. Give an example of a downside of using threads in an application? Risk of race conditions: multiple threads accessing shared data simultaneously. Example: Two threads updating a bank account balance, leading to incorrect totals. 4. What is the function of the system call fork? Creates a new child process, duplicating the parent process. Allows multitasking with separate execution. 5. Is a child process the same as a thread? No, child processes are independent with separate memory. Threads share resources and memory within the same process. 6. Can a child process live without its parent? Yes, child processes can continue running if the parent terminates. They may become orphans and are adopted by the system’s init process. 7. Can a thread live on its own? No, threads cannot exist without a parent process. They rely on the process’s resources. OSC 30  8. How can you protect a (critical) piece of code? Give the name of 2 examples. Mutex (locking mechanism): ensures exclusive access to a resource. Semaphore (signaling mechanism): controls access among multiple threads. (like a red light = you shoulndt do it but you still can) Lecture 5 Resource Allocation and Deadlocks Key Concepts of Deadlock Deadlock occurs when processes are unable to proceed because each is waiting for resources held by another, resulting in a standstill. The four essential conditions for a deadlock to occur are: Mutual Exclusion: Only one process can use a resource at a time. Hold and Wait: Processes holding resources can request additional resources. No Preemption: Resources cannot be forcibly taken from a process. Circular Wait: A cycle of processes exists, each waiting for a resource held by the next. Handling Deadlocks Methods for addressing deadlocks include: Ignoring the problem: Assuming deadlocks will not occur. Detection and Recovery: Identifying deadlocks and recovering from them, often using algorithms. Deadlock Prevention: Ensuring at least one of the four deadlock conditions cannot hold, thus preventing deadlocks from occurring. Visual Representation OSC 31  Resource Allocation Graphs: These can illustrate the state of resources and processes, helping to identify potential deadlocks. Virtual Memory Management Summary (Fastest to Slowest): Registers (in CPU)→ L1 Cache → L2 Cache → L3 Cache → RAM → SSD → HDD → External/Network Storage HDD (Hard Disk Drive) SSD (Solid State Drive) The trade-off: Speed decreases as capacity increases moving down the hierarchy. Changing values in binary file vs changing values in memory Summary of Key Differences: Aspect Changing Values in a Binary File Changing Values in Memory Changes are permanent (saved Changes are temporary (lost after Persistence to disk) program ends) Speed Slow (due to disk I/O operations) Fast (direct access) More complex (file management Complexity Simple (direct memory manipulation) needed) Data storage (databases, file Temporary computation (variables, Use Case handling) runtime data) OSC 32  MMU memory management unit #include int main() { int x = 0; // Initialize x to 0 for (int i = 0; i < 10; i++) { x++; // Increment x printf("x = %d on address %p\n", x, (void*)&x); // Print x and its address } return 0; } The memory address of x ( 0x7ffee2fdfc38 in this example) remains constant throughout the loop because x is stored in the same stack memory location during the program's execution. Running./example &./example creates two independent processes, each with its own x variable and stack memory. The memory addresses might look identical, but they are distinct because of the process isolation provided by the operating system. Introduction to Virtual Memory OSC 33  Virtual Memory allows a system to use disk space as an extension of RAM, enabling larger processes to run than may fit in physical memory. This is achieved through mechanisms like paging and segmentation. Consecutive memory allocation Memory of a processis one whole piece fixed : Static partitioning variable : Dynamic partitioning Non-consecutive memory allocation Memory of a process is divided in multiple pieces fixed : Paging variable : Segmentation Static : you determine the size of the partition DYnamic: they look at process and decide how big the partition should be Memory Management Techniques Paging: OSC 34  paging table Divides logical address space into pages and physical address space into frames. Pages are mapped to frames, aiding efficient memory usage and reducing fragmentation. Translation to Physical Address: OSC 35  The logical address (base + offset) is translated by the Memory Management Unit (MMU) into a physical address using a segment table or a page table. OSC 36  Segmentation: cannot be split up but will be rather swapped out segment table segments are swapped between disc and main memory as needed can result in memory fragmentation: a lot of small segments with gaps in between OS knows start and size of each sgment in physical memory each segment is atmoic, either the whole segment is in RAM, or none of the segment is in RAM segments can be “pushed together” to limit fragmentation and allow large segments to be loaded Divides memory into segments based on logical divisions like code, data, and stack. Provides a more flexible way of managing memory compared to paging. OSC 37  Fragmentation: Internal fragmentation is wasted space within allocated blocks due to fixed-size allocation, while external fragmentation is wasted space between blocks caused by scattered free memory. combine smaller memory parts into a bigger one so a big process can use that memory instead of being empty External fragmentation in case of dynamic partitioning The operating system decides to reorganize memory (= garbage collection) Coalescing = rebuild partition table, consecutive free partitions will be merged into one big free partition Compaction = all occupied partitions will be stored consecutively followed by one free partition; only possible in case of relocatable code OSC 38  Combination of the both ++ Performance and Efficiency Cache Memory: Acts as fast-access memory to alleviate the performance drawbacks of accessing main memory. Page Replacement Algorithms: Common algorithms include FIFO (First In, First Out), LFU (Least Frequently Used), and LRU (Least Recently Used), each with their advantages and drawbacks in managing a system’s memory. Important Takeaways Deadlock conditions must be understood and managed to ensure smooth operation of processes. Virtual memory is a crucial mechanism that allows for efficient memory usage and process management. Practical exercises enhance understanding and application of theoretical concepts in operating systems. OSC 39  Lecture 5 Q&A 1. What is deadlocking? A situation where two or more processes are unable to proceed because each is waiting for resources held by the other. 1. Who was Dijkstra? A Dutch computer scientist known for contributions to algorithms and operating systems; famous for “Dijkstra’s algorithm” for shortest paths. 2. What are the 4 deadlock conditions? Mutual Exclusion: Only one process can use a resource at a time. Hold and Wait: Processes holding resources wait for others. No Preemption(Jokio pirmumo): Resources cannot be forcibly taken. Circular Wait: A closed loop of processes, each waiting for a resource held by the next. 3. What is the difference between deadlock avoidance & deadlock prevention? Avoidance: Design system to avoid deadlock states. (there could be a deadlock but try to fix it ) Prevention: Ensure at least one deadlock condition cannot hold. (overhead) 4. What are safe sequences and why are they interesting? A sequence of process execution that guarantees all processes can finish. Interesting because it helps manage resource allocation safely. 5. What is the concept of virtual memory? An abstraction allowing programs to use more memory than physically available by using disk space as an extension. 6. What is the difference between the difference layer caches (1,2,3)? L1: Smallest, fastest, closest to CPU. L2: Larger, slower than L1, still fast. OSC 40  L3: Largest, slowest, shared among cores. 7. What is the difference between an executable and a process? Executable: Static file on disk (e.g.,.exe). Process: Dynamic instance of a program in execution, using CPU and memory. 8. What is “swapping”? Moving a process from RAM to disk (swap space) to free up memory for other processes. 9. What is partitioning, paging, segmentation in the context of memory layout strategies? Partitioning: Dividing memory into consecutive blocks. Paging: Dividing memory into fixed-size non-consecutive pages. Segmentation: Dividing memory into variable-sized segments. 10. What is trashing? Excessive paging causing performance degradation, CPU spends more time swapping than executing. 11. What is the difference between a page and a frame? Page: Fixed-size block of virtual memory. Frame: Fixed-size block of physical memory. 12. What is the copy-on-write principle? Technique where processes share resources until one modifies it, then a copy is made to maintain separation. OSC 41  cloning example Lecture 6 Lecture 6.1 Understanding File Systems File System Defined: Acts as an intermediary between the data representation managed by the kernel (e.g., /dev/sda ) and the user process (e.g., /home/user/myfile ). Performance Disparity: Due to the significant speed difference between disk access (10^-2 to 10^-3 seconds) and CPU/memory access (10^-6 to 10^-7 seconds), optimizing file system performance is crucial for any OS. Types of Storage Main Memory vs. Secondary Storage OSC 42  Main Memory: Characteristics: Small (MB/GB), expensive, volatile, directly accessible by CPU. Example: Access via virtual memory address using functions like malloc. Secondary Storage: Characteristics: Large (GB/TB), cheap, persistent, cannot be directly accessed by CPU. Data Handling: Must be loaded into main memory using commands like open. Physical Storage Types Hard Disks: Directly attached or network-attached. Use of different interfaces such as IDE and SATA. Basic geometry includes surface, track, and sector. SMART Disks: Utilize SCSI, SAN, and NAS technologies to enhance performance. Partitioning: Essential for making storage accessible to the OS. Two primary methods: Master Boot Record (MBR) and GUID Partition Table (GPT). Partitioning Techniques MBR: Limitations: Maximum of four partitions, one can be extended. Capacity: Limited to 2 TB due to 32-bit addressing. GPT: OSC 43  Modern Standard: Supports up to 128 partitions and allows for storage up to 9.7 zettabytes. Redundancy: Indexes are spread across the disk for reliability. File System Commands and Structures Command Usage: Tools like fdisk , cfdisk , and parted are employed to manage partitions. File Control Block (FCB): Each file is mapped to an FCB through a directory structure that includes file metadata such as owner, permissions, and size. INODE Structure: Each file corresponds to an INODE, which is a data structure storing information about the file. Directory Entries: Organized as pairs of , facilitating quick access. Mounting File Systems Mount Points: Locations in the directory structure that point to different storage devices. Example: /home can point to a different partition, enabling flexible storage management. Permanent Mounting: Configuration in /etc/fstab ensures that file systems are automatically mounted at boot. Options include specifying the file system type, mount point, and access permissions. Mount point is just a directory: Before mounting, it’s usually an empty folder (like /mnt/usb ). OSC 44  After mounting, that directory acts as a doorway to the contents of the storage device or file system. The data isn't copied into the system—it remains on the original device (e.g., USB or partition). The mount point just gives access to it. Special File Systems Pseudo Filesystems: Examples include /proc and sysfs , which provide interfaces for interacting with the kernel. Network File Systems (NFS): Allow remote file systems to be mounted locally, facilitating data sharing across networks. Key Takeaways File systems serve as essential bridges between hardware storage and user processes in UNIX environments. Effective partitioning and mounting strategies enhance data accessibility and system performance. Special filesystems provide additional functionalities, including kernel interaction and network-based storage solutions. Lecture 6.1 Q&A 1. What does it actually mean when we say data is “persistent”? OSC 45  Data remains stored even when power is off. Example: Files on a hard drive vs. volatile memory (RAM). Persistence ensures data can be retrieved later. 2. Why does a userland process(an application running in user space) need to use a file system to store/read data on a storage medium(usb, hdd, ssd, sd) (and not just direct IO access)? File systems organize and manage data efficiently. They provide access control and metadata management. Example: Easier navigation through directories rather than raw data access. 3. When talking about non-SSD discs, what is the advantage of sequential I/O? Sequential I/O minimizes seek time, enhancing speed. Example: Reading a large video file in one continuous read vs. jumping around the disk. More efficient for large files. 4. Is this advantage the same for SSD discs? Less significant; SSDs have uniform access times. Random access speed is often close to sequential speeds. Example: SSDs can quickly access any data block. 5. Compare the two “versions” of partitioning (number of partitions, BIOS support, max disk size). MBR: 4 primary partitions, 2 TB max, BIOS support. GPT: 128 partitions, 9.7 ZB max, UEFI support. GPT is more modern and robust. 6. Why not throw everything into one single monster partition? Limits flexibility and increases risks (e.g., data loss). OSC 46  Example: Home directories can grow uncontrollably; swap space management is needed. Easier to manage and backup smaller partitions. 7. What does /dev/sda3 mean? Indicates the third partition on the first SCSI/SATA disk. Example: Used for specific data storage like a user’s home directory. 8. Why can/should you give a hex code of a partition type to a partition when you create one? Identifies the file system type for the OS. Example: Hex codes determine whether the partition is NTFS, FAT32, etc. Helps the OS manage partitions correctly. 9. What form does “a swap” have in Linux? Swap can be a dedicated partition or a file on the file system. Example: Uses disk space for overflow memory when RAM is full. 10. Which are the most used file system flavors in current Linux? Common file systems: Ext4, XFS, Btrfs, ZFS. Example: Ext4 is widely used for general purposes due to stability and performance. 11. What does it mean when something is mounted on a certain directory location? A file system is accessible at that directory. Example: Mounting a separate disk at /home allows user files to be stored there. 12. What is a pseudo file system? Virtual file systems not linked to physical storage. Example: /proc provides information about system processes and kernel parameters. OSC 47  13. The tty command tells you which terminals you are currently using. Let’s say one user gets as a reply /dev/tty1 and another user gets as a reply /dev/pts/0. What is the difference between those two users? /dev/tty1: Direct console terminal (physical). /dev/pts/0: Pseudo-terminal for remote access (like SSH). Reflects different access methods. 14. What is an NFS type partition? Network File System; allows remote file access as if local. Example: Mounting a directory from another machine onto the local file system. 15. What does /etc/fstab do? Lists file systems and how they should be mounted at boot. Example: Defines devices, mount points, and options for auto-mounting. 16. Some directories are present in almost every Linux distro. Name them and describe their role briefly. /bin: Essential system binaries. /etc: Configuration files. /home: User directories. /var: Variable data like logs. 17. What is a FCB? File Control Block; holds metadata about a file. Example: Contains information like file size, ownership, and permissions. 18. How is a directory also a file? Directories store data about other files (like a database). Example: Contains a list of filenames and their associated INODE numbers. 19. What is an INODE? Data structure that stores file metadata. OSC 48  Example: Includes file size, permissions, and pointers to data blocks. 20. What is the difference between a hard and a soft link? Hard link: Direct pointer to an INODE (same data block). Soft link: Pointer to another filename; can point to a non-existent file. 21. What is a superblock? Contains metadata about the entire file system. Example: Stores size, free blocks, and INODE information. Critical for file system management. Go more into spacial partition types Lecture 6.2 Archiving: Combining multiple files into one. OSC 49  Compression: Reducing file size by removing redundancy. Archiving vs. Compression Archiving Collapses multiple files into one for easier management. Useful for keeping related files together, such as backups or project data. Compression Makes files smaller for efficient storage and transfer. Can be applied to single files or entire directories. Uses of Archiving and Compression Managing Log Files: Combining logs for easier access and analysis. Sharing Groups of Files: Facilitating the transfer of multiple files. Project Documents: Keeping related files organized. Source Code: Bundling code files for distribution. Efficient Transfer: Reducing data size for quicker internet transmission or tape backup. Grouping by Time: Organizing files based on creation or modification dates. Lossless vs. Lossy Compression Lossless Compression The decompressed file is identical to the original. Not as efficient as lossy compression but vital for preserving data integrity. Common uses include logs, documents, binaries, and configuration files. Lossy Compression The decompressed file may lose some original information. Efficient for media files like images, sound, and movies. OSC 50  Compression Algorithms Gzip vs. Bzip2 Gzip: Utilizes Lempel-Ziv coding for lossless compression; good efficiency. Bzip2: Employs Burrows-Wheeler block sorting, slightly more efficient than gzip but requires more CPU power. Compression Techniques Lempel-Ziv: Dictionary-type compression that builds a dictionary as it reads the source data, ideal for text compression. Burrows-Wheeler: A more complex algorithm that enhances compression efficiency. Lecture 6.3 Key Vocabulary: User Accounts: Individual accounts for users to access the system. Password Placeholder: A field indicating where password information is stored. UID (User ID): A unique identifier for each user account. GID (Group ID): A unique identifier for user groups. System Accounts: Accounts for services rather than human users. Sudo Command: A command allowing users to run programs with the privileges of another user. User Accounts and Passwords The configuration of user accounts is primarily managed using files located in the /etc directory, particularly the /etc/passwd and /etc/shadow files. /etc/passwd File: Each line corresponds to a user account. Fields include: OSC 51  name: Username password placeholder: Indicates password storage (often an asterisk for security) UID: User ID GID: Primary group ID comment: Description of the user home directory: User’s home path shell: Default shell for the user /etc/shadow File: Contains password-related account information. Fields include: name: Username password: Encrypted password hashed lastchange: Last password change date min/max: Password aging settings warn/inactive/expire: Warning, inactivity, and expiration settings Viewing Account Information Account information can be retrieved using various commands: Grep Command: To find specific user details, e.g., grep sysadmin /etc/passwd. Getent Command: An alternative for displaying account details. ID Command: To verify user identity and display their UID and GID. System Accounts and Groups System Accounts: Designed for system services, typically with UIDs between 1-499. Non-login shells and asterisks in the password field of /etc/shadow. OSC 52  Deleting a system account should be done with caution. Group Accounts: Users can belong to multiple groups. The primary group is defined in /etc/passwd, while supplemental groups are in /etc/group. /etc/group File: Fields include: group_name: Name of the group password_placeholder: Group password (if applicable) GID: Group ID user_list: Members of the group Changing Groups To switch to a secondary group, users can employ the newgrp group_name command, which opens a new shell with the new primary group. Verification can be done using the id command, and users can return to the previous shell using exit. Working with Root Logging in directly as root is discouraged due to security risks. Instead, the su and sudo commands provide safer alternatives. Su Command: Opens a new shell as a different user. Example: su user1 to switch to user1. Use su - user1 for a full login session. Sudo Command: Allows execution of single commands as another user. Requires the user’s password and must be configured in the /etc/sudoers file. OSC 53  Using the Who and W Commands Who Command: Displays currently logged-in users. Example output shows user details and login times. W Command: Provides detailed information about logged-in users and system activity. Displays user ID, idle time, and what commands are being executed. Key Points User management is essential for system security. Use specific commands to view and manage user information. System and group accounts serve different purposes and require careful management. Using sudo and su enhances security while allowing necessary permissions. Lecture 6.3 Q&A 1. What can you find in the /etc/passwd & /etc/shadow files? /etc/passwd: User account details (name, UID, GID, home directory, shell). /etc/shadow: Password info (hashed password, last change, expiration details). 2. What is so special about system accounts in Linux? Designed for services, UIDs 1-499, non-login shells, critical for system. Deleting requires certainty they’re not needed. 3. What are groups in Linux? Collections of users, defined in /etc/group. Provides permissions and resource access. 4. How many groups can a user be a member of/use at once? A user can belong to multiple groups, defined in /etc/group. 5. What is the correct way of becoming root? OSC 54  Avoid direct root login. Use su --login or sudo. 6. Describe the sudo concept. Executes commands as another user. Prompts for user’s password, configured in /etc/sudoers. 7. Who’s on your system right now? Use who for logged-in users. Use w for detailed info (user activity, idle time). Lecture 7 Lecture 7.1 Ownership: The relationship between a file and its user/group. User Owner: The individual who creates a file. Group Owner: The primary group associated with the file. Commands: Tools to view and change ownership. Viewing Ownership Using ls Command To view file ownership, the ls -l command is employed. The output displays several details, including the permissions, number of links, user owner, and group owner. For example: Command: [sysadmin@localhost ~]$ ls -l /etc/named.conf Output: -rw-r-----. 1 root named 1163 May 13 10:27 /etc/named.conf OSC 55  For directories, the ls -ld command provides similar information but focuses on the directory itself rather than its contents. Command: [sysadmin@localhost ~]$ ls -ld /etc/named Output: drwxr-x---. 2 root named 4096 Mar 28 2013 /etc/named Using stat Command The stat command offers a more detailed view of file attributes, including access rights, user ID (UID), and group ID (GID). Command: [sysadmin@localhost ~]$ stat /etc/named Key output includes: Access Rights: (0750/drwxr-x---) Uid: (0/root) Gid: (25/named) File Ownership Every file in a Linux system is owned by a user and a group. The user who creates a file becomes its user owner, while the primary group of that user becomes the group owner. Changing Ownership: The chown command allows the root user to change file ownership. The chgrp command enables users to change the group owner of their files. OSC 56  Orphaned Files When a user or group is deleted, their files may still show the original UID or GID, leading to orphaned files. This affects access and management. Identity Information Finding Your Identity To understand your current user identity and group memberships, the id command can be executed. Command: [sysadmin@localhost ~]$ id Example Output: uid=500(sysadmin) gid=500(sysadmin) groups=500(sysadmin),1 0001(research),10002(development) Viewing Group Membership The groups command lists all groups a user belongs to. Command: [sysadmin@localhost ~]$ groups sysadmin Example Output: sysadmin research development Changing File and Group Ownership The newgrp Command The newgrp command allows users to switch their primary group for file creation purposes. This command requires knowledge of the group password. OSC 57  Users can revert to their original group using the exit command. The chgrp Command Users can use the chgrp command to change the group ownership of their files. The root user can change any file’s group owner. Example usage: chgrp groupname The chown Command While primarily for root users, the chown command can also be used by regular users to change their file’s group owner. Conclusion Understanding file ownership is integral to managing security and access in Linux systems. The commands ls , stat , chown , chgrp , and newgrp provide the necessary tools to view and modify file ownership. Proper management of file permissions not only secures data but also ensures that users have the appropriate access levels to their files and directories. As system administrators apply these concepts, they enhance overall system integrity and user accountability. Main Takeaway: Effective management of file ownership enhances system security and user permissions. Key Commands: ls , stat , chown , chgrp , newgrp Implications: Understanding and utilizing file ownership is essential for maintaining a secure and efficient Linux environment. Lecture 7.1 Q&A 1. Which command statements could you use to determine the ownership of a file/directory in Linux? ls -l : To view ownership of a regular file. ls -ld : To view ownership of a directory. OSC 58  stat : For detailed ownership info. 2. Explain ownership levels in Linux. User Ownership: Owner of the file. Group Ownership: Group associated with the file. 3. Explain CHOWN. What does it do? Who can use it? When? Changes file user/group ownership. Can be used by root. Used when changing file ownership. 4. Explain CHGROUP. What does it do? Who can use it? When? Changes file group ownership. Users can change their files’ group; root can change any file’s group. Used when a user needs to reassign group ownership. 5. What is an orphaned file? A file with a deleted or changed user UID/group GID, showing former UID/GID as owner. 6. Which command statements could you use to determine your user’s identity/group memberships? id : Shows user ID and group memberships. whoami : Displays current user. groups : Lists group memberships. 7. Explain NEWGRP. What does it do? Who can use it? When? Changes effective primary group for the user. Users can use it if they know the group password. Used before creating a file to set the group context. Lecture 7.2 OSC 59  Understanding File Permissions File Type and Permissions: The first character of the ls -l output indicates the file type. Characters 2-4: Permissions for the user owner. Characters 5-7: Permissions for the group owner. Characters 8-10: Permissions for others. Permission Application: Only one of the three sets of permissions applies during access attempts: If the user is the file owner, the user owner permissions apply. If the user is part of the group owning the file, the group owner permissions apply. If neither, the others permissions apply. Example Command: ls -l /etc/passwd Output: rw-r--r--. 1 root root 4135 May 27 21:08 /etc/passwd Interpretation: The user owner (root) has read and write permissions, the group owner (root) has read permissions, and others have read permissions. OSC 60  BOB cannot even access the file because he cannot traverse to the directory because he isnt in the root group and also doesnt have execute permiossion on other groups Using chmod Command Symbolic Method: Syntax: chmod [who][operator][what] Examples: chmod u+x abc.txt : Adds execute permission for the user owner. chmod go-rx abc.txt : Removes read and execute permissions for the group and others. OSC 61  Numeric Method: Permissions represented by three-digit octal numbers: 4 = read, 2 = write, 1 = execute. Combined values range from 0 to 7. Examples: chmod 755 abc.sh : Grants rwxr-xr-x permissions. chmod 660 abc.txt : Grants rw-rw---- permissions. Understanding umask The umask value defines default permissions for newly created files and directories. OSC 62  Maximum allowable permissions: Files: rw-rw-rw- (666) Directories: rwxrwxrwx (777). Example: User with a umask of 002 results in group and others lacking write permission. Root with a umask of 022 removes write permission for group and others. Setting umask: Display current value: umask Change value: umask 027 Extending Permissions: Access Control Lists (ACLs) Need for ACLs: Simplifies access control for specific processes (e.g., a web server) without exposing files to the entire world. Enabling ACLs: Requires mounting the filesystem with the acl option. Setting ACLs: Use setfacl to modify ACLs. Verify changes with getfacl. Special Permissions Setuid Permission Allows users to execute programs with the permissions of the file owner (often root). Example: The passwd command can update the /etc/shadow file. OSC 63  Indicated in ls -l with a lowercase s in the user execute position: rwsr-xr-x indicates effective setuid. Setgid Permission Similar to setuid but applies to group permissions. Example: The wall command allows messages to be sent to all terminal users. Indicated by lowercase s in the group execute position: r-xr-sr-- indicates effective setgid. Sticky Bit Permission Prevents users from deleting files they do not own in a shared directory (e.g., /tmp). Indicated by a lowercase t in the execute position for others: drwxrwxrwt. Lecture 7.2 Q&A 1. Which three permissions can a file have? Read ®: Allows viewing file content. Write (w): Permits modifying file content. Execute (x): Enables running the file as a program. 2. What are the three levels in which permissions are set? User (u): Permissions for the file owner. Group (g): Permissions for users in the file’s group. Others (o): Permissions for all other users. 3. Consider this: “-rw-r–r--. 1 root root 4135 May 27 21:08 /etc/passwd”. What does the first - mean in this case? What else could you see there? The first indicates it’s a regular file. Other possibilities: d for directory, l for symbolic link. OSC 64  Next characters show permissions: rw- (user), r-- (group), r-- (others). 4. How do set permissions result in effective permissions for a given user in Linux? Effective permissions depend on ownership: If the user is the owner, user permissions apply. If the user is in the group, group permissions apply. If neither, “others” permissions apply. 5. Explain CHMOD. What does it do? Who can use it? When? Changes file/directory permissions. Used by file owner or root. Used when needing to modify access rights. 6. What is the difference between using CHMOD symbolically or numerically? Symbolically: Uses letters (u, g, o) and operators (+, -, =). Example: chmod u+x file.txt (adds execute permission for user). Numerically: Uses octal values (0-7) for permissions. Example: chmod 755 file.txt (sets rwxr-xr-x permissions). 7. Explain UMASK. What does it do? Who can use it? When? Sets default permissions for newly created files/directories. Used by all users. Applied when files/directories are created, influencing base permissions. 8. Explain ACL. What does it do? Who can use it? When? Access Control Lists (ACLs) provide more granular permission control. Used by users with appropriate privileges (root or file owner). Useful when needing to grant specific permissions beyond standard owner/group/others. 9. Explain SETUID. What does it do? Who can use it? When? OSC 65  Allows a file to run with the privileges of the file owner. Set by root or file owner. Useful for allowing non-privileged users to execute certain commands with elevated privileges. 10. Explain SETGUID. What does it do? Who can use it? When? Allows a file to run with the privileges of the group owner. Set by root or file owner. Useful for enabling group access for specific commands or files. 11. Explain the sticky bit. What does it do? Who can use it? When? Prevents users from deleting files they do not own in shared directories. Set by directory owners or root. Commonly used in directories like /tmp to enhance security while allowing file sharing. Lecture 7.3 Firmware Stage The Firmware Stage is the initial part of the boot process, traditionally associated with the Basic Input Output System (BIOS). In modern systems, the Unified Extensible Firmware Interface (UEFI) has largely replaced BIOS, although the term “BIOS” is still commonly used. 2 Primary jobs in this stage: 1. Power-On Self Test (POST): This process ensures that critical hardware components such as the CPU, RAM, and peripherals are functioning correctly. 2. Load Master Boot Record (MBR): The MBR contains the drive’s partition table and loads the first stage bootloader, which is responsible for starting the next stage of the boot process. First Stage Boot Loader OSC 66  The First Stage Boot Loader is crucial for locating the MBR on the first detected hard drive and loading the bootloader. Size limitations: The MBR is limited to 446 bytes, with 64 bytes allocated for the partition table and 6 bytes for the CRC. Common Bootloaders: LILO (Linux Loader): Supports systems with BIOS. ELILO (EFI Linux Loader): Supports UEFI systems. GRUB (Grand Unified Bootloader): Versatile and supports kernel flavor choices, UEFI, and legacy BIOS. Other examples include SILO (SPARC Improved BootLoader) for SPARC hardware and YABOOT for PowerPC systems. PXE (Preboot Execution Environment): Allows for network booting using TFTP to load a bootloader from a server. GRUB2 Boot.img : The first stage bootloader, under 512 bytes, points to the location of core.img. Core.img: This second stage bootloader loads the chosen kernel image and necessary modules from the /boot/grub directory. Kernel Phase During the Kernel Phase, GRUB2 loads the selected kernel into memory and transfers control to it. OSC 67  Initial RAM Disk Image: An initrd/initramfs is utilized as temporary storage. SystemD Activation: This marks the beginning of the operating system’s main initialization process. SystemD Phase In the SystemD Phase, several critical tasks are performed: Kernel options are set using /etc/sysctl.conf. The udevd daemon detects all connected devices. The system imports network configurations and performs filesystem checks (fsck) on the root filesystem if necessary. It handles encrypted filesystems and mounts filesystems as per /etc/fstab. The system eventually boots into a specific target mode. Systemctl Command The systemctl command is essential for managing services in systems that use SystemD. Examples of commands: Start a service: # systemctl start httpd.service Stop a service: # systemctl stop httpd.service Check service status: # systemctl status httpd.service View all running services: # systemctl -a Enable a service to start automatically: # systemctl enable httpd.service Daemons and Targets OSC 68  Daemons: These are background processes that provide services to the system. Targets: A target represents a specific state of the system where a defined number of services are running. It can have required, parallel, conflicting, and sequential dependencies. Lecture 7.3 Q&A 1. In which stages can you split up a system’s boot process? Four main stages: Firmware Stage, First Stage Bootloader, Kernel Phase, SystemD Phase. Some stages modifiable by admins. 2. What is the difference between firmware/bios/uefi? Firmware: Software for hardware control. BIOS: Older firmware standard. UEFI: Modern replacement for BIOS, more features. 3. What is a bootloader? Software that loads the operating system. Initiates the boot process after firmware. 4. Bootloaders are usually called 2-(or multi)stage bootloaders. Why? They operate in stages; first stage loads second stage, which then loads the OS or kernel. 5. How does GRUB2 “boot” a system? GRUB2 loads kernel and initial RAM disk into memory, passing control to the kernel. 6. What does SystemD actually “do”? Manages system startup, services, and targets; initializes system components post-boot. 7. Which commands/statements could you use to troubleshoot/monitor a booting system? dmesg , to view kernel messages; OSC 69  Check /var/log/messages and /var/log/syslog ; Use systemctl commands for service management and status. Lecture 8 Lecture 8.1 Understanding Firmware Definition of Firmware: Firmware is a specialized layer of software that facilitates communication between hardware components and the operating system. Functionality: It initializes hardware and transfers control to the OS. Control Transfer: The firmware hands over control to a “payload,” which can either be a bootloader like GRUB or a minimal version of the Linux kernel. Coreboot: An Open-Source Alternative Introduction to Coreboot: Coreboot is an open-source firmware project that replaces standard BIOS or UEFI, allowing for more flexibility and control. Payload Options: Users can choose different payloads that coreboot can load, including various bootloaders or lightweight kernels, enabling customized boot configurations. Building Custom Firmware Setting Up: Begin by cloning the coreboot repository from https://review.coreboot.org/coreboot in a Linux VM. Toolchain Requirement: Before building firmware, a toolchain is necessary. A toolchain comprises programming tools used for software development, primarily consisting of a compiler and a linker. Diversity in Toolchains: Different Linux distributions may include distinct toolchains. Coreboot has its own toolchain, which can be built using the command make crossgcc-i386. OSC 70  Navigating Coreboot Source Code Source Code Structure: 3rdparty: Contains third-party software needed for hardware initialization. configs: Holds predefined configurations for building coreboot. Documentations: Includes specific documents about various mainboards and components. payloads: Contains resources for building different payloads. src: The most crucial directory, containing all source code for coreboot. Emulating Your Firmware Using QEMU: QEMU is a Type 2 hypervisor that excels at emulating a generic x86 PC, allowing for testing and development of the custom firmware. Configuration Steps: Use make menuconfig to configure coreboot for a generic PC. Set up the desired payloads, such as SeaBIOS for primary boot or CoreInfo for hardware information. Building and Running Custom Firmware Compilation Process: After configuration, run make to build a coreboot.rom image, which can take time. Execution: Start the emulated generic PC using the command qemu-system- x86_64 -bios build/coreboot.rom -serial stdio. Customizing Payloads and Flashing Firmware Cross Compilation: Users can cross-compile their desired open-source BIOS for any compatible hardware. Running Applications: The firmware can be configured to run games like Doom or Tetris or directly launch a kernel or GRUB. Flashing Process: Once satisfied, the custom firmware can be flashed onto actual hardware, allowing users to control their system’s boot process. OSC 71  This chapter encapsulates the essence of firmware, coreboot, and the process of creating custom solutions, fostering a deeper appreciation for the intricacies of system boot processes. Lecture 8.1 Q&A 1. Why would you replace your firmware with an open source version? Control over features Security transparency Customizability Community support 2. What is a "payload" in the coreboot world? Component that takes control after firmware Examples: Bootloader like GRUB, stripped-down Linux kernel 3. What is a toolchain? Set of programming tools for software development Includes compilers and linkers Coreboot uses a custom toolchain (crossgcc-i386) 4. When you use Coreboot to build your own BIOS ROM, are all black boxes gone then? No, some components may still be proprietary Open source firmware reduces unknowns 5. What is QEMU? Type 2 hypervisor for Linux Emulates generic x86 PC for testing firmware 6. What is Seaboot? Primary payload in coreboot Open source BIOS implementation OSC 72  7. Give some secondary payload examples. CoreInfo (shows hardware info) Running games like Doom or Tetris 8. How can you build a custom firmware ROM for other hardware? Cross-compile open source BIOS for supported hardware Choose secondary payloads, then flash firmware to hardware Gain control over system booting Lecture 8.2 SystemD: A system and service manager for Linux. Daemons: Background processes managed by SystemD. systemctl: Command-line tool to interact with SystemD..service files: Configuration files for services. The Role of SystemD SystemD is fundamentally in control of the Linux operating system’s service management. When you run systemctl without arguments, it provides a comprehensive overview of the current active services, showcasing SystemD’s monitoring capabilities. Active Services: SystemD lists all running services. Control: SystemD dictates service initiation and dependencies. Service Configuration with.service Files Each service managed by SystemD is defined within a.service file. These files enable SystemD to understand what actions to take regarding that service. Unit Section: Defines the service’s metadata. DESCRIPTION: A human-readable title for the service. AFTER: Specifies that the service should start after another service. OSC 73  BEFORE: Indicates that the service should start before another specified service. Service Execution Parameters Within the Service Section of a.service file, several critical parameters dictate how a service operates: ExecStart: Command executed when the service starts. ExecReload: Optional command for service restart. Type: Defines the startup type, with options like simple, forking, and oneshot. Restart: Specifies conditions under which the service should restart (options include on-failure and always). Install Section and Service Enabling The Install Section outlines how to enable or disable services using systemctl enable and systemctl disable. This section is crucial for ensuring that services are automatically started during system boot. WantedBy: Indicates the target for which the service should be started. RequiredBy: Defines hard dependencies, ensuring services fail if dependencies are not met. Creating and Managing Your Own Services Creating a custom service involves creating a script and corresponding.service file. For instance, a script named update-on-boot.sh can be placed in /root/ , allowing system updates at boot time. Script Execution: Ensure the script is executable and tested. Service Creation: In /etc/systemd/system , create a file like update-on-boot.service. Example of a Custom Service An example service file might look like this: [Unit] Description=Keeping my sources minty fresh After=multi -user.target [Service] ExecStart=/usr/bin/bash /root/update-o OSC 74  n-boot.sh Type=simple [Install] WantedBy=multi-user.target Commands: Use systemctl daemon-reload to refresh SystemD’s awareness of services, followed by enabling the service. User Services For user-specific services, the configuration directory is ~/.config/systemd/user/. Here, users can create their own service files without requiring root privileges. User Service Example: A service logging uptime can be created, which triggers at shutdown. Building and Managing a Chatbot Service For more complex applications, such as a chatbot, a dedicated.service file can manage its execution. The components include: Service Definition: Includes parameters for automatic restarts and execution commands. Service Management: Use systemctl --user commands to manage user services. [Unit] Description=Dumb chat service After=network.target [Se rvice] Type=simple Restart=always ExecStart=/usr/bin/env php %h/chatbot.php [Install] WantedBy=default.target Lecture 8.2 Q&A 1. What does systemD "do" when you get down to it? Manages system services and daemons. Controls the starting, stopping, and monitoring of services. Uses the systemctl command to interact. 2. Which command allows you to communicate with SystemD? systemctl command. 3. What are.service files (for)? OSC 75  Configuration files for services. Define how systemD should manage a service. 4. Explain the following keywords: Description: Human-readable service title. Before: Starts current service before specified service. After: Starts current service after specified service. ExecStart: Command to execute when starting the service. ExecReload: Command for service restart. Restart: Defines when to restart service (e.g., on-failure). WantedBy: Specifies dependencies for runlevels. RequiredBy: Specifies hard dependencies, failure leads to service failure. 5. How can you make systemD aware of the presence of a new service? Use systemctl daemon-reload command. 6. What happens when you "enable" a service that is wanted by a Target? Creates a symlink in the target's wants directory. The service starts automatically when the target is reached. 7. What is the difference between a user-level.service and a root-level.service? User-level services run in the user's session. Root-level services run system-wide, affecting all users. 8. How is the home directory specified differently in a script vs a.service file? In scripts: use $HOME. In.service files: use %h for user home directory. 9. What's the advantage of a SystemD service over a scheduled or a user- started process? Automatic management and respawn of processes. OSC 76  Better dependency handling and integration with system states. 10. What happens when you kill a process started via SystemD? The process is respawned automatically if configured to do so (e.g., Restart=always ). Lecture 9 Lecture 9.1 Windows Operating System Overview Pros of Windows: Easier setup and configuration, leading to a lower learning curve for users. Better technical support, as Windows remains prevalent in corporate environments. Seamless integration with other Microsoft software (e.g., SQL Server, Exchange, Office 365). Comprehensive feature set, catering to a wide range of user needs. Cons of Windows: Less reliable for servers, resulting in higher downtime. Being the most widely used OS makes it a primary target for malware and scammers, raising security concerns. Higher Total Cost of Ownership (TCO) due to licensing fees. Often perceived as less performant, requiring more system resources compared to alternatives. Software Deployment Software deployment refers to the process of putting applications into production so that end-users can access them. OSC 77  Over the past decade, user expectations have shifted towards demanding 100% uptime and a preference for web applications. The popularity of deploying on Windows Servers has declined but remains a viable option. Windows Client vs. Windows Server Windows 10 and Windows Server 2016/2019 share a similar GUI and are built on the same NT kernel. Key differences include: Higher hardware support in Windows Server (more memory and CPUs). Support for many more concurrent remote connections in Windows Server compared to Windows 10. Windows Server can be installed without a GUI through Windows Server Core, enhancing security and resource efficiency. Know the LTSC and SAC OSC 78  Command-Line Tools Distinctions between Cmd and PowerShell: Cmd outputs in text, while PowerShell outputs in objects. PowerShell provides advanced functionalities such as cmdlets and extensive scripting capabilities. Key commands: Get-Process for process management. lusrmgr.msc for user management. eventvwr.msc for logging/events. Windows Server Editions and Installation Options Windows Server comes in multiple editions, including Standard and Datacenter, with the latter offering unlimited virtualization rights. Installation options include: Server Core: Minimal installation, no GUI. Server with Desktop Experience: Full GUI, user-friendly. Nano Server: Designed for cloud applications and containers. OSC 79  Windows Licensing Methods 1. Retail Licensing For individuals or small businesses. Transferable to another device. Activated via a product key. 2. Volume Licensing For organizations needing multiple licenses. Managed via KMS (local server, periodic reactivation) or MAK (limited activations). Cost-effective for bulk purchases. 3. OEM Licensing Pre-installed on devices by manufacturers (e.g., Dell, HP). Tied to the hardware, non-transferable. Activated via BIOS/UEFI. Server Management Management can be conducted locally or remotely using tools such as RDP (Remote Desktop Protocol) and Windows Admin Center. Management interfaces include: Traditional GUI tools like Server Manager and Microsoft Management Consoles (MMCs). Command-line tools such as PowerShell and legacy commands. OSC 80  2 of the same active directory why? ⇒ incase one server is done nothing happens the other takes over = “high availability” OSC 81 OSC 82 OSC 83 OSC 84 OSC 85 OSC 86  intra= within and between PCs inter= between different sites Active Directory and User Authentication Active Directory (AD) serves as the backbone for user and resource management in Windows environments. OSC 87  Key features of AD include: Domain-based authentication using Kerberos. Organizational Units (OUs) for managing groups of users and applying specific policies. AD replication for synchronizing changes across domain controllers. User Authentication: Domain-based authentication allows users to log in from any machine in the domain. Local authentication is restricted to the machine where the user account is created. Security Principles and Encryption The CIA Triad (Confidentiality, Integrity, Availability) forms the foundation of security principles in Windows environments. Encryption methods: Private (symmetric) key cryptography vs. Public (asymmetric) key cryptography. Examples of encryption standards include AES for symmetric encryption and RSA for asymmetric encryption. Key Takeaways: Effective deployment strategies and server management practices can lead to enhanced performance and security. Knowledge of Active Directory, user authentication, and encryption is vital for maintaining secure Windows environments. Round Robin DNS = load balancing technique which can be used for dns, o