OS fundamentals - 8 - storage management.pdf

OS FUNDAMENTALS Storage management 1 Phaedra Degreef GENERAL OVERVIEW • Storage device − • Partition − − • Block-level access Fixed-size section of storage device A bit vintage … ..replaced by volumes now Logical volumes Volumes − Volume group • − Pool of storage Logical volume • •...

OS FUNDAMENTALS Storage management 1 Phaedra Degreef GENERAL OVERVIEW • Storage device − • Partition − − • Block-level access Fixed-size section of storage device A bit vintage … ..replaced by volumes now Logical volumes Volumes − Volume group • − Pool of storage Logical volume • • Filesystems, swap areas, db storage Like a partition Filesystem − 2 Organisation of files and directories Partitions RAID arrays Volume groups Storage devices STORAGE DEVICES STORAGE DEVICES • Hardware or network service (!) − Magnetic devices − SSD devices • sectors/tracks ➔ blocks of data • Low-level formatting defines blocks (on software- level) − Done at factory − Don’t try this at home (BIOS) → time consuming, unnecessary − Rather: zero-filling to erase data − SATA: secure erase (multiple writes) 4 MAGNETIC HARD DISK STORAGE 5 MOUNTING DEVICES • In Linux, (all) physical devices are represented by a device file in /dev • These are created by the system as disks are discovered: − − − − /dev/sda /dev/sdb /dev/sdc /dev/sdd • Later, these will need to be “mounted” 6 PARTITIONS PARTITIONING • Lowest level of storage management • Two types − Traditional partitioning AKA MBR − GUID partition table (GPT) / EFI 8 MBR PARTITIONING TRADITIONAL PARTITIONING • Disk starts with a “label” that identifies the partition • Label is written at the beginning of the disk − Together with other information, depending on OS • A partition table (also at the beginning of the disk) stores information about all partitions on disk − On PC hardware, partition table must be on boot disk 10 MBR • Master Boot Record − • Starts at sector 0 − • Aka “boot record” 512 bytes long Contains partition table and bootloader 11 PARTITION TABLE IN MBR Max 4 partitions • Size limit 2TB per disk • 2 types of partitions: • − Primary partition − Extended partition • Can contain another partition table 12 EXTENDED PARTITION • Extended partition contains another partition table − Refers to “logical partitions” − These do not exist in the main MBR! − That’s why some systems have trouble with them 13 MBR PARTITIONING • Typically, extended partition in the last partition on the disk • Some OS (Windows) can not boot from extended partition • System boots from “Active” partition (Windows) − Only 1 partition can be active 14 GPT PARTITIONING EFI / GPT PARTITIONING • Extensible Firmware Interface (EFI) − − − − Overcome MBR limit of 2TB per disk Only 1 type of partition Identified by GUID ➔ GUID partition table (GPT) Backward compatible with MBR • By setting an MBR in the first block of the partition table − Windows can only boot from GPT disks if the EFI firmware is present on the hardware − Linux and Mac OS do not have this limitation 16 PARTITIONING SCHEME IN LINUX Filesystem layer /home /opt /spare Partition layer /dev/sda1 /dev/sda2 /dev/sdb1 Physical layer 17 Hard disk 1 Hard disk 2 /dev/sda /dev/sdb 18 WINDOWS PARTITIONING SCHEME 19 LOGICAL VOLUME MANAGEMENT 20 LOGICAL VOLUME • Abstraction of a partition • Flexible, dynamic: volumes can shrink or grow − Adapt to changing storage space needs − Partitions are very rigid (fixed in size) • Logical Volume Management − Create/delete volumes − Grow/shrink volumes − Move volumes (between physical devices) 21 LINUX LVM LVM2 is latest incarnation • Volume groups • − = collection of physical volumes − Physical volumes are added to a volume group − Logical volumes are created within a volume group 22 STORAGE MANAGEMENT ADDING A DISK TO YOUR SYSTEM • Step 1: partition the new disk • Step 2: format the partition(s) • Step 3: mount the partition(s) 24 ADDING A DISK TO YOUR SYSTEM • Step 1: partition the new disk − Your first disk will be called /dev/sda • If you added a second disk, it will be /dev/sdb • Etc… − fdisk command is used to partition the disk • Partitions will be named » /dev/sdb1 » /dev/sdb2 » Etc…. 25 Example: +500M will give you a 500MB partition ADDING A DISK TO YOUR SYSTEM • Step 2: format the partition(s) − This will create a filesystem on your partition(s) • What is a filesystem? − store files in a tree structure of directories. • Organisation of grouping any number of sectors into one file, • organizing files into directories, and • setting access rights 26 ADDING A DISK TO YOUR SYSTEM • Step 2: format the partition(s) − Linux supports many filesystems in the kernel • ext2 is considered the standard filesystem • ext3 added journaling to ext2 • ext4 is the latest update − Command to use: mkfs (“make file system”) • mkfs –t filetype /dev/DEVICE_PARTITION • mkfs.filetype /dev/DEVICE_PARTITION • Example: » mkfs.ext3 /dev/sdb1 » mkfs –t ext3 /dev/sdb1 27 ADDING A DISK TO YOUR SYSTEM • Step 3: mount the partition(s) − The new partition now has a filesystem − That filesystem will become a part of the host filesystem • It will be added or “plugged in” to the existing tree structure 28 ADDING A DISK TO YOUR SYSTEM • Example: removable disks − /dev/hdb has directories ‘Redhat’ and ‘dosutils’ on it − It will be mounted on directory /mnt/cdrom • The “mount point” − Directories on cdrom will now be accessible as • /mnt/cdrom/Redhat • /mnt/cdrom/dosutils 29 MOUNTING FILESYSTEMS • command: mount − mount <partition> <mount-point> − sudo mount /dev/sda1 /mnt/temp • Mounts partition sda1 to /mnt/temp • All files and directories of the partition will be visible under directory /mnt/temp − When logging out or shutting down, it will be unmounted again! • Special case: swapon command − Determines which partition the system will use for swapping and paging − Must be on contiguous space! 30 MOUNTING FILESYSTEMS • Manual mouting (mount command) − Not persistent: at end of session, partition is unmounted again − Not mounted again in next session • Automatic mounting − At start of each new session − Partitions to be mounted are configured in file /etc/fstab (“filesystem table”) − Be very careful with changes in this file • If there are errors in it, your system will not start • Test contents of file with mount –a command 31 MOUNTING FILESYSTEMS • Automatic mounting on startup: /etc/fstab file 32 /ETC/FSTAB EXAMPLE 33 34 SOME USEFUL STORAGE COMMANDS • Disk space usage: df (-h option gives a more user-friendly formatting): 35 SOME USEFUL STORAGE COMMANDS • Show disks and partitions: lsblk 36 SOME USEFUL STORAGE COMMANDS • More info on disks and partitions: fdisk –l 37 SOME USEFUL STORAGE COMMANDS • Disk usage statistics per (sub-)directory: du 38 WINDOWS VOLUME MANAGEMENT WINDOWS VOLUME MANAGEMENT Concept of disk / partition / volume is similar to Linux • No volume groups • Use of drive letters for partitions • 40 DISK TYPES • Basic disk • Dynamic disk 41 BASIC DISK • Legacy type • MBR or GPT partitioning (if h/w supported) • Basic volume = partition on a basic disk − Primary partition − Logical drives in extended partition (MBR) − limited to one physical device 42 DYNAMIC DISK • Allows creation of volumes that can span more than 1 disk − − − − 43 Simple volume Spanned volume Striped volume Mirrored volume SIMPLE VOLUME • Similar to a basic partition • No size limit 44 SPANNED VOLUME • Uses 2 – 32 (max) dynamic disks • Spanned volume is a merge of parts on different disks 45 STRIPED VOLUME • Spanned volume, but all chunks on different disks are equal in size − RAID 0 • Data to be written will be divided into 64 KB blocks that will be written in parallel to all disks • Maximizes read/write performance 46 MIRRORED VOLUME • Data is written to two equal-size dynamic disks • Redundancy • Fault tolerance • Less storage space efficiency double of space required) 47 RAID RAID • Redundant Array of Inexpensive Disks • Distribute or replicate data across multiple disks • Can be implemented in hardware (RAID controller) or software (not necessarily slower) • Presents abstraction: OS sees one logical drive 49 RAID LEVELS • • • • • • • 50 Linear or JBOD RAID 0 RAID 1 RAID 10 RAID 5 RAID 50 RAID 6 RAID 0 • Striping 51 RAID 1 • Mirroring 52 RAID10: RAID 0+1 • Combination − Striping − Mirroring of striped disks 53 RAID 10: RAID 1+0 • Combination − Mirror disks − Write stripes to mirrored disks 54 RAID 5 • • • • 55 Striping + distributed parity Effective storage capacity = N-1 disks Parity allows recalculation of lost block One disk can fail without any data loss RAID 5 56