File Management Lecture Notes PDF

Chapter 8 Understanding Operating Lecture 9 & 10: File Management  Learning Objectives After completing this chapter, you should be able to describe: The fundamentals of file management and the structure of the file management system File-naming c...

Chapter 8 Understanding Operating Lecture 9 & 10: File Management  Learning Objectives After completing this chapter, you should be able to describe: The fundamentals of file management and the structure of the file management system File-naming conventions, including the role of extensions The difference between fixed-length and variable-length record format Understanding Operating Systems, Sixth Edition 2  Learning Objectives (cont'd.) The advantages and disadvantages of contiguous, noncontiguous, and indexed file storage techniques Comparisons of sequential and direct file access Access control techniques and how they compare The role of data compression in file storage Understanding Operating Systems, Sixth Edition 3  Brief introduction on what is File Manager (by OpenCanvas). Watch this video before you move to the next slide. https://www.youtube.com/watch?v=yY9lY6C_8vg Understanding Operating Systems, Sixth Edition 4  The File Manager File management system – Software File access responsibilities – Creating, deleting, modifying, controlling Support for libraries of programs – Online users Spooling operations Interactive computing Collaborates with device manager To understand more on File Management, click on https: https://www.youtube.com/watch?v=MbWBm6rxCUk https://www.youtube.com/watch?v=T3cnK8vK2lA 5 Understanding Operating Systems, Sixth Edition  Responsibilities of the File Manager Four tasks – File storage tracking – Policy implementation Determine where and how files are stored Efficiently use available storage space Provide efficient file access – File allocation if user access cleared Record file use – File deallocation File returned to storage Communicate file availability Understanding Operating Systems, Sixth Edition 6  Responsibilities of the File Manager (cont'd.) Policy determines: – File storage location – System and user access Uses device-independent commands Access to material – Two factors Flexibility of access to information (Factor 1) – Shared files – Providing distributed access – Allowing users to browse public directories Understanding Operating Systems, Sixth Edition 7  Responsibilities of the File Manager (cont'd.) Subsequent protection (Factor 2) – Prevent system malfunctions – Security checks Account numbers and passwords File allocation – Activate secondary storage device, load file into memory, update records File deallocation – Update file tables, rewrite file (if revised), notify waiting processes of file availability Understanding Operating Systems, Sixth Edition 8  Definitions Field – Group of related bytes – Identified by user (name, type, size) Record – Group of related fields File – Group of related records – Information used by specific application programs Report generation – Flat file No connections to other files, no dimensionality Understanding Operating Systems, Sixth Edition 9  Definitions (cont'd.) Databases – Groups of related files – Interconnected at various levels Give users flexibility of access to stored data Program files – Contain instructions Directories – Listings of filenames and their attributes Understanding Operating Systems, Sixth Edition 10  Definitions (cont'd.) Understanding Operating Systems, Sixth Edition 11  Interacting with the File Manager User Commands – OPEN, DELETE, RENAME, COPY Device independent – Physical location knowledge not needed Cylinder, surface, sector – Device medium knowledge not needed Tape, magnetic disk, optical disc, flash storage – Network knowledge not needed Understanding Operating Systems, Sixth Edition 12  Interacting with the File Manager (cont'd.) Understanding Operating Systems, Sixth Edition 13  Interacting with the File Manager (cont'd.) Logical commands – Broken into lower-level signals – Example: READ Move read/write heads to record cylinder Wait for rotational delay (sector containing record passes under read/write head) Activate appropriate read/write head and read record Transfer record to main memory Send flag indicating free device for another request Performs error checking and correction – No need for error-checking code in programs Understanding Operating Systems, Sixth Edition 14  Typical Volume Configuration Volume – Secondary storage unit (removable, nonremovable) – Multifile volume Contains many files – Multivolume files Extremely large files spread across several volumes Volume name – File manager manages – Easily accessible Innermost part of CD, beginning of tape, first sector of outermost track Understanding Operating Systems, Sixth Edition 15  Typical Volume Configuration (cont'd.) Understanding Operating Systems, Sixth Edition 16  Typical Volume Configuration (cont'd.) Master file directory (MFD) Stored immediately after volume descriptor Lists – Names and characteristics of every file in volume File names (program files, data files, system files) – Subdirectories If supported by file manager – Remainder of volume Used for file storage Understanding Operating Systems, Sixth Edition 17  Typical Volume Configuration (cont'd.) Single directory per volume – Supported by early operating systems Disadvantages – Long search time for individual file – Directory space filled before disk storage space filled – Users cannot create subdirectories – Users cannot safeguard their files – Each program needs unique name Even those serving many users Understanding Operating Systems, Sixth Edition 18  Introducing Subdirectories File managers – Create MFD for each volume Contains file and subdirectory entries – Improvement over single directory scheme Problems remain: unable to logically group files Subdirectory – Created upon account opening – Treated as file Flagged in MFD as subdirectory Unique properties Understanding Operating Systems, Sixth Edition 19  Introducing Subdirectories (cont'd.) File managers today – Users create own subdirectories (folders) Related files grouped together – Implemented as upside-down tree Efficient system searching of individual directories May require several directories to reach file Understanding Operating Systems, Sixth Edition 20  Introducing Subdirectories (cont'd.) Understanding Operating Systems, Sixth Edition 21  Introducing Subdirectories (cont'd.) File descriptor – Filename: ASCII code – File type: organization and usage System dependent – File size: for convenience – File location First physical block identification – Date and time of creation – Owner – Protection information: access restrictions – Record size: fixed size, maximum size Useful information on file system: https://www.youtube.com/watch?v=mzUyMy7Ihk0 Understanding Operating Systems, Sixth Edition 22  File-Naming Conventions Filename components – Relative filename and extension Complete filename (absolute filename) – Includes all path information Relative filename – Name without path information – Appears in directory listings, folders – Provides filename differentiation within directory – Varies in length One to many characters Operating system specific Understanding Operating Systems, Sixth Edition 23  File-Naming Conventions (cont'd.) Extensions – Appended to relative filename Two to three characters Separated by period Identifies file type or contents – Example BASIA_TUNE.MP3 – Unknown extension Requires user intervention Understanding Operating Systems, Sixth Edition 24  File-Naming Conventions (cont'd.) Understanding Operating Systems, Sixth Edition 25  File-Naming Conventions (cont'd.) Operating system specifics – Windows Drive label and directory name, relative name, and extension – UNIX/Linux Forward slash (root), first subdirectory, sub-subdirectory, file’s relative name Understanding Operating Systems, Sixth Edition 26  File Organization Arrangement of records within files All files composed of records Modify command – Request to access record within a file Understanding Operating Systems, Sixth Edition 27  Record Format Fixed-length records – Direct access: easy – Record size critical – Ideal for data files Variable-length records – Direct access: difficult – No empty storage space and no character truncation – File descriptor stores record format – Used with files accessed sequentially Text files, program files – Used with files using index to access records Understanding Operating Systems, Sixth Edition 28  Record Format (cont'd.) Understanding Operating Systems, Sixth Edition 29  Physical File Organization Describes: – Record arrangement – Medium characteristics Magnetic disks file organization – Sequential, direct, indexed sequential File organization scheme selection considerations – Volatility of data – Activity of file – Size of file – Response time Understanding Operating Systems, Sixth Edition 30  Physical File Organization (cont'd.) Sequential record organization – Records stored and retrieved serially One after the other – Easiest to implement – File search: beginning until record found – Optimization features may be built into system Select key field from record and sort before storage Complicates maintenance algorithms Preserve original order when records added, deleted Understanding Operating Systems, Sixth Edition 31  Physical File Organization (cont'd.) Direct record organization – Uses direct access files – Direct access storage device implementation Random organization Random access files – Relative address record identification Known as logical addresses Computed when records stored, retrieved – Uses hashing algorithms to transform a key field Understanding Operating Systems, Sixth Edition 32  Physical File Organization (cont'd.) Direct record organization (cont'd.) – Advantages Fast record access Sequential access if starting at first relative address and incrementing to next record Updated more quickly than sequential files No preservation of records order Adding, deleting records is quick – Disadvantages Hashing algorithm collision Similar keys Understanding Operating Systems, Sixth Edition 33  Physical File Organization (cont'd.) Understanding Operating Systems, Sixth Edition 34  Physical File Organization (cont'd.) Indexed sequential record organization – Best of sequential and direct access – ISAM software: creates, maintains – Advantage: no collisions (no hashing algorithm) Generates index file for record retrieval Divides ordered sequential file into equal sized blocks Each entry in index file contains the highest record key and physical data block location Search index file – Overflow areas To understand more about types of File Organization (with example), click on : https://www.youtube.com/watch?v=bvoJI7WKBbc for part 1 and https://www.youtube.com/watch?v=LYLXKdD6n2Y for part 2 35 Understanding Operating Systems, Sixth Edition  Physical Storage Allocation File manager works with files – As whole units – As logical units or records Within file – Records must have same format – Record length may vary Records subdivided into fields Application programs manage record structure File storage – Refers to record storage Understanding Operating Systems, Sixth Edition 36  Physical Storage Allocation (cont'd.) Understanding Operating Systems, Sixth Edition 37  Contiguous Storage Records stored one after another – Advantages Any record found once starting address, size known Easy direct access – Disadvantages Difficult file expansion, fragmentation For more example, click on: https://www.youtube.com/watch?v=W5_NWxUpDLM Understanding Operating Systems, Sixth Edition 38  Noncontiguous Storage Files use any available disk storage space File records stored in contiguous manner – If enough empty space Remaining file records and additions – Stored in other disk sections (extents) – Extents Linked together with pointers Physical size determined by operating system Usually 256 bytes Understanding Operating Systems, Sixth Edition 39  Noncontiguous Storage (cont'd.) File extents linked in two ways – Storage level Each extent points to next one in sequence Directory entry Filename, storage location of first extent, location of last extent, total number of extents (not counting first) – Directory level Each extent listed with physical address, size, pointer to next extent Null pointer indicates last one Understanding Operating Systems, Sixth Edition 40  Noncontiguous Storage (cont'd.) Advantage – Eliminates external storage fragmentation – Eliminates need for compaction Disadvantage – No direct access support Cannot determine specific record’s exact location Understanding Operating Systems, Sixth Edition 41  Noncontiguous Storage (cont'd.) Understanding Operating Systems, Sixth Edition 42  Noncontiguous Storage (cont'd.) For more example, click on: https://www.youtube.com/watch?v=j8iSwq35xGU Understanding Operating Systems, Sixth Edition 43  Indexed Storage Allows direct record access – Brings pointers together Links every extent file into index block Every file has own index block – Disk sector addresses for file – Lists entry in order sectors linked Supports sequential and direct access Does not necessarily improve storage space use Larger files experience several index levels Understanding Operating Systems, Sixth Edition 44  Indexed Storage (cont'd.) For more example, click on: https://www.youtube.com/watch?v=lzAY_mxFDAI Understanding Operating Systems, Sixth Edition 45  More explanation on file allocation method by DexTutor, click on https://www.youtube.com/watch?v=Yj0iBwijs4g Understanding Operating Systems, Sixth Edition 46  Lecture 10: PART 2 Understanding Operating Systems, Sixth Edition 47  Access Methods Dictated by a file organization Most flexibility: indexed sequential files Least flexible: sequential files Sequential file organization – Supports only sequential access Records: fixed or variable length Access next sequential record – Use address of last byte read Current byte address (CBA) – Updated every time record accessed Understanding Operating Systems, Sixth Edition 48  Access Methods (cont'd.) Understanding Operating Systems, Sixth Edition 49  Sequential Access Update CBA Fixed-length records – Increment CB CBA = CBA + RL Variable-length records – Add length of record (RL) plus numbers of bytes used to hold record to CBA CBA = CBA + N + RL Understanding Operating Systems, Sixth Edition 50  Direct Access Fixed-length records (RN: desired record number) – CBA = (RN – 1) * RL Variable-length records – Virtually impossible Address of desired record cannot be easily computed – Requires sequential search through records – Keep table of record numbers and CBAs Indexed sequential file – Accessed sequentially or directly – Index file searched for pointer to data block Understanding Operating Systems, Sixth Edition 51  Levels in a File Management System Understanding Operating Systems, Sixth Edition 52  Levels in a File Management System (cont'd.) Level implementation – Structured and modular programming techniques – Hierarchical Highest module passes information to lower module Modules further subdivided – More specific tasks Uses information of basic file system – Logical file system transforms record number to byte address Understanding Operating Systems, Sixth Edition 53  Levels in a File Management System (cont'd.) Verification at every level – Directory level File system checks if requested file exists – Access control verification module Determines whether access allowed – Logical file system Checks if requested byte address within file limits – Device interface module Checks if storage device exists Understanding Operating Systems, Sixth Edition 54  Access Control Verification Module File sharing – Data files, user-owned program files, system files – Advantages Save space, synchronized updates, resource efficiency – Disadvantage Need to protect file integrity – Five possible file actions READ only, WRITE only, EXECUTE only, DELETE only, combination – Four methods Understanding Operating Systems, Sixth Edition 55  Access Control Matrix Advantages – Easy to implement – Works well in system with few files, users What is ACM: https://www.youtube.com/watch?v=jyNU4aZTwrc 56 Understanding Operating Systems, Sixth Edition  Access Control Matrix (cont'd.) Disadvantages – As files and user increase, matrix increases Possibly beyond main memory capacity – Wasted space: due to null entries Understanding Operating Systems, Sixth Edition 57  Access Control Lists Modification of access control matrix technique Implementing Access Control: https://www.youtube.com/watch?v=kXzNqBT_QK4 Understanding Operating Systems, Sixth Edition 58  Access Control Lists (cont'd.) Contains user names granted file access – User denied access grouped under “WORLD” Shorten list by categorizing users – SYSTEM Personnel with unlimited access to all files – OWNER Absolute control over all files created in own account – GROUP All users belonging to appropriate group have access – WORLD All other users in system Understanding Operating Systems, Sixth Edition 59  Capability Lists Lists every user and files each has access to Can control access to devices as well as to files Most common Understanding Operating Systems, Sixth Edition 60  Data Compression A technique used to save space in files Text decompression Other decompression schemes Understanding Operating Systems, Sixth Edition 61  Text Compression Records with repeated characters – Repeated characters are replaced with a code Repeated terms – Compressed using symbols to represent most commonly used words – University student database common words Student, course, grade, department each be represented with single character Front-end compression – Entry takes given number of characters from previous entry that they have in common Understanding Operating Systems, Sixth Edition 62  Other Compression Schemes Large files – Video and music ISO MPEG standards – Photographs ISO – International Organization for Standardization Understanding Operating Systems, Sixth Edition 63  Summary File manager – Controls every file and processes user commands – Manages access control procedures Maintain file integrity and security – File organizations Sequential, direct, indexed sequential – Physical storage allocation schemes Contiguous, noncontiguous, indexed – Record types Fixed-length versus variable-length records – Four access methods Understanding Operating Systems, Sixth Edition 64