OS: Resource Management and Kernels

Resource Management & Abstraction

• The OS manages hardware such as CPU, RAM, and storage
• Abstraction simplifies intricacies so programs can interface with resources without needing specific details
• Primary benefit is portability, allowing programs to function across diverse hardware setups
• Security is increased through abstraction because it restricts direct access to hardware, thereby lowering risks
• Efficiency is reached as the OS manages resource allocation dynamically

Preemptive Multitasking

• The OS can interrupt a process to switch to another
• Ensures that one process does not monopolize the system

Monolithic Kernel

• All operations (device drivers, memory, file system) operate in Kernel Mode(Ring 0)
• Faster processing and fewer situation switches are achieved
• The entire system is at risk as a single failure can cause it to crash

Microkernel

• Runs essential functions (IPC, scheduling, memory) in Kernel mode
• Other services operate in user space
• Increased modularity provides more security because failures do not cause whole system crashes
• Slower than monolithic kernels because of increased IPC overhead

FUSE/udev/sshfs

• Services migrate to user space in microkernel designs
• Users can create file systems without kernel modifications with FUSE (Filesystem in Userspace)
• Device nodes are dynamically managed in user space with udev
• Remote file system mounting over SSH without kernel modifications is allowed via sshfs

Command-Line Interface

• The command line is powerful and adaptable, but has a steeper learning curve

Text User Interface

• Menu-driven interfaces like ncurses programs such as htop

Graphical User Interface

• Graphical desktops are user-friendly but require resources

Kernel Mode (Ring 0)

• Has unrestricted hardware access
• Runs key OS functions like memory and device drivers

User Mode

• It has limited access, is only able to make system calls to interact with hardware
• Maintains system stability by preventing direct hardware access

Security

• Separation is enforced by the CPU based on privilege levels

Process

• A running instance of a program

PID

• Unique identifier assigned to each process by the OS

Program Binaries

• A program binary (e.g., /bin/ls) exists as a file.
• Transforms into a process with its own PID, memory, and state when executed.

Process States

• Processes move through these stages: New → Ready → Running → Waiting (I/O) → Terminated

Process Symbols

• Tools like ps assign representational letters to states (e.g., R for running, S for sleeping)

Address Space Layout

• Each process is assigned a virtual address space
• Executable instructions are housed in the code segment
• Global/static variables are stored in the data segment
• Dynamic memory such as malloc in C uses the Heap
• Function calls and local variable are managed by the Stack
• Command-line arguments and environment variables are stored by Arg/env

User ID (UID)

• Every user including root is assigned one
• Root (UID = 0) has total control
• Only a software label controlled by the OS

File

• The basic unit of data within storage

File Systems

• A structure that gives files organization (e.g. ext4, NTFS)
• Provides abstraction for storage devices like HDD, SSD, and USB drives

Library Calls (lib calls)

• Functions offered via libraries such as printf(), malloc()
• Run in user space

System Calls (sys calls)

• Direct requests made to the OS itself, such as open(), read(), write()
• Mandate a movement into kernel space

Lib Calls vs Sys Calls

• Lib calls are streamlined because they remain in user space
• Sys calls come with overhead but offer straight access to hardware

Abstraction

• Elevated levels of abstraction enhance software portability across diverse OS and hardware setups
• Code written in C is workable for Windows and Linux
• Assembly (low-level) is CPU-specific

Environment Variables

• Key-value pairs configure shell or process, e.g., $PATH, $HOME, $USER

Global vs. Local Variables

• Global variables are exported (available to all child processes)
• Local variables are only available to the current shell

Internal (Built-in) Commands

• Executed directly by the shell such as cd, echo, export

External Commands

• Distinct binaries reside in system directories such as /bin/ls, /usr/bin/cat
• Internal commands execute faster because they don't need a separate process

Direct Execution

• Running code directly on CPU, like kernel code or native processes

Indirect Execution

• Utilizing an interpreter like Python scripts or Java bytecode

Limited Direct Execution

• The OS allows controlled program execution by using interrupts (such as system calls, preemptive multitasking)

Execution Context

• The state of a process
• Comprises registers, memory, open files

Context Switch

• CPU changes from one process to another
• The state of current process is saved and the next one is loaded
• Involves overhead because saving/restoring registers and TLB flushes (expensive)

Scheduling

• Determines how the CPU allocates time
• Common scheduling strategies are FIFO (First-In-First-Out, simple but unfair), Round Robin (each process gets a time slice), Priority Scheduling (higher priority run first), Multilevel Queue (varied queues for system or user)

fork()

• This creates a new process duplicating the parent

exec()

• Replaces the process memory with a brand new program

forking and exec

• A child thread can alter state before calling exec() allowing current process replacement
• fork() enhances parallel execution

Processes vs Threads

• Processes have distinct memory allocation
• Threads share the memory

Faster Context Switching

• Threads provide faster context switching with reduced overhead

Easier Communication

• Another perk of threads is easier communication through shared memory

Race Conditions

• A threat of threads
• Occurs when multiple threads alter data at the same time

pthreads (POSIX threads)

• High-level threading API that functions across OS

clone()

• Creates a process or thread using a low-level system call
• Tougher to manage compared to pthreads, offers greater control

Paging

• Divides memory into fixed-size pages and maps to physical memory utilizing a page table

Swapping

• When memory runs full, entire processes are moved from RAM to disk
• Greater data movement renders this slower than paging

Kernel in Abstraction

• Offers a simpler interface for hardware, execution, memory, and to manage system access

Static Memory Allocation

• Occurs during compile time with fixed sizes and variable declarations

Stack Memory Allocation

• Memory allocation occurs during runtime of functions and local variables

Heap Memory Allocation

• Memory allocation used for dynamic allocation malloc/free, new/delete

Memory Symbols

• Functions, variables are represented
• Can be seen using nm on a binary file
• Symbols can exist just at compile-time or at runtime

Library Calls

• Lib calls operate in user mode and use functions from libraries e.g. printf(), malloc()

System Calls

• Syscalls trigger a context switch into kernel mode (added overhead)
• Syscalls directly request OS services like open(), read(), write()

Memory Access

• Library calls are faster and syscalls involve more overhead and provide direct access to hardware

Library Functions

• At compile time library functions are resolved and linked

Dynamic Linking

• Shared libraries are loaded by dynamic linking at load time

Actual Execution

• Actual execution and system call transitions to kernel mode occur during runtime

Terminal

• /sbin/getty getty starts the processes when a terminal is opened waiting for login

Login

• /usr/bin/login Handles user authentication and reads user credentials from /etc/passwd and /etc/shadow

Shell

• /bin/bash Custom shell like 3000shell starts after login and proceeds to interpret user commands"

User Account Information

• /etc/passwd holds usernames, UIDs, GIDs, home directories, shell paths
• /etc/shadow stores hashed passwords (more secure)

UID

• (User ID) identifies a user

EUID

• (Effective UID) determines privileges
• With setuid elevated privileges can be enabled, e.g., sudo

GID

• (Group ID), EGID (Effective GID): Like UID, but for groups

Octal Notation

• To show representation such as: rwxr-xr-- → 754

Symbolic Notation

• To change permissions of an owner/group using: chmod u+x file

Special permission bits

• setuid (chmod u+s) to carry out commands as file's owner
• setgid (chmod g+s) to carry out commands as file group
• sticky bit (chmod +t) restrict file deletions to only the directory owner

The Shell Workflow

• Reads input and then it parses and tokenizes commands
• Expands wildcards and variables such as $HOME
• Determines if build-in (cd, export) or external (ls, grep)
• A child process is forked, when external
• It replaces process memory with exec() and waits for the process completion

Init process

• init (PID 1), the first process launched by the kernel
• Manages system initialization and background services
• Employs modern systems like systemd, SysV init, or OpenRC

Zombie Process

• When a process ends its execution with an entry in the process table
• Resulting process calls for the parent process using a wait() function to clean up

Zombie Reaping

• The OS then gets rid of the zombie when wait() is called

Signals

• Asynchronous notifications which are sent to processes e.g. SIGKILL, SIGSTOP
• No extra data or payload is transferred pre-defined signal number

Signal Concurrency

• Issue of signal loss, if it is sent before processing is ready

Signal Handling

• SA_RESTART ensures interrupted system calls are reattempted to prevent failure

Signal Actions

• sigaction() sets a signal, and kill(pid, SIGTERM) sends termination signal

Shell Pipeline

• | passes stdout of one command to stdin of the next such as cat file.txt | grep "error" | wc -l"

Redirection

• < file reads input, > file overwrites, and >> file appends output

Input Provision

• Methods include command-line args, piped, and standard input

File Location

• Path signifies a file's pathway, the exact name, location
• Filename points to a file name excluding the path
• Absolute Pathname is the full path starting from root
• Relative Pathname starts with the current directory versus root

File Commands

• open() command Opens a file descriptor
• read()/write() Commands that can send data to another file descriptor
• ioctl() sends control commands such as changing terminal configurations

Mounting a File System

• Mounting means attaching a filesystem such as a USB, or system drive to a target directory
• mount point can be tested and configured

Shells

• Command-line interpreters that facilitate user interactions with OS
• Examples are bash, zsh, fish, and sh

Shell Commands

• Two types of commands internal and external

Shell Features

• Provides features like variables, scripting, pipes, redirection, and job control

Streams

• Common streams are /dev/stdout, /dev/stdin, and /dev/stderr

Device Files

• Rep standard process from the system

/dev/stdin

• Represents the standard input file descriptor 0

/dev/stdout

• Represents the standard output file descriptor 1

/dev/stderr

• Represents the standard error file descriptor 2

Login Process

• starts by invoking getty (or agetty) to trigger, show login prompt
• the system hands control /usr/bin/login process
• Credentials are cross checked using /etc/passwd and /etc/shadow processes
• Starts shell upon validating authentication status, e.g./bin/bash

User Account

• Basic user information (except passwords) /etc/passwd stores the credentials

Environment

• Consists of of parameters that influence shell processes
• $HOME shows user home directory

RAM

• Fast, volatile memory and storage HDD/SDD non-volatile, slower

Allocation of Memory

• I/0 operations refer to read/write data between RAM,Storage

Storage in Blocks

• Each piece of data managed in blocks, smallest unit of disk I/O
• Block device layer, buffering reads/writes operations

Files store Inodes

• Two types of system files; regular files and named pipes
• Metadata includes but not limited to file types, permissions, owner, size
• Metadata is located across the data

File Descriptors

• Each process tracks open files via integer ID

File System Structure

• Includes the metadata of a file system with pointers

Directories and Data

• maps a filename to an inode number that contains data

Links

• Two distinct links exist, hard links vs soft links

Hard Files

• Hard name for a single file multiple names pointing to a file

Device Files

• Virtual File System VFS not a physical disk exist, specific files only

Block vs Character files

• Block device files store fixed size data /dev/sda /dev/loop0
• Character files store char data such as /dev/random, dev/tty

File Properties

• Logical size the amount of data
• Physical size the actual space

Logical Size Comparisons

• 1 > P then a sparse file
• P > 1 involves disk fragmentation

System Crash

• Journaling files e.g. ext4, ensuring that all system data are consistent and working even if the unexpected should occur

System Recovery

• Runs fsck and will repair
• Has data recovery + hardware replacement options

OS Information

• Special files does not provide data, or store it
• User space has the ability create customized file system types if desired

Connections: Fragmentation

• Is split into Unusable chunks

Memory and Storage

• Memory uses pages, while storage uses blocks

Hardlinks

• All dependencies are independent of the binary

Softlinks

• Allows systems to saves time since they load into drive time

Attributes managed by OS

• Environment variables, file descriptor
• Relative paths determine whether a file refers to current

Listing Commands

• List the content directory and its parts

stat

• Used when diagnosing files

df and du

• Used when reporting level and utilization size, or identifying space utilization