File Systems: Storage and Management

File Systems: Essential Requirements for Long-Term Storage

• Essential for storing large amounts of information for extended durations.
• Ensures data persists even after the process using it has ended.
• Facilitates concurrent access to information by multiple processes.

Thought Experiment: Storing Data Without External Management

• Explore how a single program could directly manage data storage on a device.
• Consider the methods to achieve this and potential use cases.

Disk as a Linear Sequence of Blocks

• Visualize a disk as a contiguous series of fixed-size blocks.
• The two fundamental operations are reading a specific block, denoted as 'k', and writing to block 'k'.

Key Questions Arising in File System Design

• How is specific information located on the disk?
• How can the system prevent unauthorized access to a user's data by another user?
• How does the system determine which disk blocks are available for use?

File Naming Conventions

• File extensions indicate the file's content or type, such as:
  • .bak for Backup files
  • .c for C source code
  • .gif for Compuserve Graphical Interchange Format images.
  • .hlp for Help files
  • .html for World Wide Web HyperText Markup Language documents
  • .jpg for still pictures encoded with the JPEG standard
  • .mp3 for audio files encoded in MPEG layer 3
  • .mpg for movie encoded with the MPEG standard
  • .o for object files that are the output of a compiler
  • .pdf for Portable Document Format files
  • .ps for PostScript files.
  • .tex for Input for the TEX formatting program
  • .txt General text file
  • .zip for Compressed archives

File Structure Types

• Byte Sequence: Treating a file as a continuous stream of bytes.
• Record Sequence: Structuring a file as a series of records.
• Tree Structure: Organizing a file's contents in a hierarchical tree format.
• General-purpose Operating Systems commonly utilize the byte sequence structure.

File Types

• Executable files include a header with magic number, text size, data size, BSS size, symbol table size, entry point, and flags.
• Archive files have a module name, date, owner, protection settings, and size information within their header.

Defining File Attributes

• Protection: Specifies who has access rights and the type of access allowed.
• Password: A password needed to grant access to the contents of the file.
• Creator: The ID of the creating user
• Owner: The owner to which the file belongs
• Read-only flag: Set to 0 for read/write access or 1 for read-only.
• Hidden flag: If set to 1 the file is not displayed in directory listings
• System Flag: Specifies (0) normal or (1) a system-level file
• Archive flag: Set to 0 if the file has been backed up or 1 if it needs to be backed up
• ASCII/binary flag: set to 0 for ASCII format or 1 for binary
• Random access flag: sequential (0) or random access only (1)
• Temporary flag: (0) Normal or (1) delete upon process exit
• Lock flags: 0 unlocked, nonzero if locked
• Record Length: Number of bytes in a record
• Key Position: Offset of the key within each record
• Key Length: Number of bytes in the key field
• Creation time: Records the date and time when the file was created.
• Time of last access: Records the date/time the file was last used
• Time of Last Change: Date and time of the last file data modification
• Current Size: The files current size
• Maximum size: The maximum file size

Common File Operations

• Create: Generates a new, empty file.
• Delete: Removes a file from the file system.
• Open: Prepares a file for reading or writing.
• Close: Finalizes file access, often writing buffered data to the disk.
• Read: Transfers data from a file into memory.
• Write: Transfers data from memory into a file.
• Append: Adds data to the end of a file.
• Seek: Moves the file pointer to a specific position within the file.
• Get attributes: Retrieves metadata associated with a file.
• Set attributes: Modifies metadata, such as permissions or timestamps, of a file.
• Rename: Changes the name of a file.

File System Copy Program

• A file copy program involves opening an input file and creating an output file.
• It reads blocks of data from the input, and writes them to the output file, continuing until the end of file.
• Includes the closing files and reporting errors, ensuring efficient copying with minimal overhead.

Single-Level Directory Systems

• Basic structure consisting of a directory entry and four files.

Structure of Hierarchical Directory Systems

• A tree-like structure featuring a root directory and user directories to organize files.
• Includes user subdirectories and distinct user files for structured storage.

Directory Operations

• Create: Constructs a new, empty directory.
• Delete: Removes a directory, contingent on it being empty.
• Opendir: Sets up a directory for reading its contents.
• Closedir: Concludes directory access.
• Readdir: Obtains directory entries.
• Rename: Modifies the name of a directory.
• Link: Creates a connection between two files or directories.
• Unlink: Deletes a link, potentially removing referenced data.

Analyzing a general File System Layout

• Partition Table: Provides a disk map, indicating partition locations.
• MBR (Master Boot Record): Initiates system boot-up.
• Boot Block: Facilitates OS loading.
• Superblock: Contains key file system metadata.
• Free Space Management: Manages available space on the device
• I-nodes: Index nodes for file information.
• Root Directory: The base of the file system.
• Data and Directories: Where available data are stored

File Implementation: Contiguous Layout

• Contiguous allocation allocates continuous disk space for files, simplifying access.
• Benefit: Simplicity and enhanced performance.
• Drawback: The disk state after D and F files have been removed.

Storing Files: Implementing Linked List Allocation of Disk Blocks

• Linked list allocation method manages data
• This method allows non-contiguous storage via pointers in each disk block.

File Allocation Table in Memory

• Linked list allocation can use a file allocation table stores in main memory

Files Implementation: I-nodes

• I-nodes store file metadata and addresses of the first few data blocks directly.
• Includes a pointer to a disk block containing additional disk addresses, allowing for larger files.

Simple Directories

• Fixed-size entries hold disk addresses and attributes directly in the directory entry.
• Alternative approach: directory entries refer only to i-nodes, which contain attributes.

Methods for Managing Long Filenames

• In-line: Directly storing full filenames within the directory entry reduces fragmentation.
• In a heap: Store the content elsewhere

UNIX File Removal Protocol

• Remove file's name from its corresponding directory.
• Release the specific i-node back into the pool.
• Return the occupied data blocks to the pool of available, free disk blocks.

Virtual File System Role

• Acts as an interface between the user process level and the underlying file systems and buffer cache layers.
• Includes VFS interface, FS1, FS2, and FS3

VFS Data Structures and Code

• Process Table: manages running processes
• File descriptors: Index into file table for open files
• V-nodes: Virtual nodes; an in-memory object that represents a file or directory
• Function pointers to write, read, open
• Read functions, provides Call from VFS and FS 1

Disk Space Management

• Data shows a comparison between allocation strategies by file size in bytes
• Data given by percentage

Keeping Track of Free Blocks (1)

• Store the free list on a linked list
• Use a bitmap

Disk Quotas

• Quotas are tracked on by a per user basis in the quota table
• The quota table includes, soft block and file limits, hard block and file limits, current usage, warnings left

Backing up to tape

• Done to handle Recoveries from disaster or stupidity
• Main competing goals are Security of Data and Reasonbleness of speed

File System Consistency

• The states of file system:
• consistent, missing block, duplicate block in free list, duplicate data block

Reducing Disk Arm Motion

• I-nodes are located near the start of the disk
• Disk is divided into cylinder groups, each with its own i-nodes.

MS-DOS File System

• Includes Filename Extension, Attributes, Reserved data, Time and Date stamps and block number
• Has data for block size vs block number
• Data provided for FAT-12, FAT-16 and FAT-32

UNIX V7 directory entry

• Includes bytes, filename and I-node number

A UNIX i-node

• includes Attributes and single, double, and triple indirect blocks
• points to disk addresses and Addresses of data blocks

The steps in looking up /usr/ast/mbox

• Root directory to is for/usr, to Block that is to I-node is for / usr/ast to Block of /ast/usr directory
• /usr/ast is i-node

ISO 9660 directory entry.

• Includes extended attribute record length, and directory entry length
• Flags and interleave
• Base file name and System version

Joliet Extensions

• These extensions include support for long file names and a Unicode character set to accommodate a wider range of characters.
• Enables directory nesting deeper than eight levels, allowing for more complex directory structures.
• Introduces the use of directory names with extensions, adding flexibility in naming conventions.