Chapter5(1) (2).ppt

Chapter 5: The Filesystem 1  Outline  Introduction  Pathnames  Filesystem Mounting and Unmounting  Organization of the File Tree  File Types  File Attributes  Access Control Lists  Working with Files  Working with File Contents...

Chapter 5: The Filesystem 1  Outline  Introduction  Pathnames  Filesystem Mounting and Unmounting  Organization of the File Tree  File Types  File Attributes  Access Control Lists  Working with Files  Working with File Contents 2  Introduction  The filesystem is used as a rendezvous point to connect clients with the drivers they are seeking.  The basic purpose of a filesystem is to represent and organize the system’s storage resources.  The filesystem can be thought of as comprising four main components:  A namespace - a way to name things and organize them in a hierarchy  An API - a set of system calls for navigating and manipulating objects  Security model - schemes for protecting, hiding, and sharing things  An implementation - software to tie the logical model to the hardware 3 Introduction 4  Pathnames  The filesystem is presented as a single unified hierarchy that starts at the directory / and continues downward through an arbitrary number of subdirectories.  / is also called the root directory.  Graphical user interfaces often refer to directories as “folders,” even on Linux systems. Folders and directories are exactly the same thing  The list of directories that must be traversed to locate a particular file plus that file’s filename form a pathname. 5  Pathnames (1)  Pathnames can be either  absolute (e.g., /tmp/foo)  or relative (e.g., book4/filesystem).  Relative pathnames are interpreted starting at the current directory  The terms filename, pathname, and path are more or less interchangeable  Filename and path can be used for both absolute and relative paths; pathname usually suggests an absolute path. 6  Pathnames (2)  The filesystem can be arbitrarily deep.  However, each component of a pathname (that is, each directory) must have a name no more than 255 characters long.  There’s also a limit on the total path length you can pass into the kernel as a system call argument (4,095 bytes on Linux).  To access a file with a pathname longer than this,  you must cd to an intermediate directory and use a relative pathname. 7  Filesystem Mounting and Unmounting  The filesystem is composed of smaller chunks  called filesystems  each of which consists of one directory and its subdirectories and files.  The term “file tree” to refer to the overall layout  reserve the word “filesystem” for the branches attached to the tree.  Some filesystems live on disk partitions or on logical volumes backed by physical disks.  filesystems can be anything that obeys the proper API: a network file server, a kernel component, a memory-based disk emulator, etc. 8  Filesystem Mounting and Unmounting (1)  Most kernels have a nifty “loop” filesystem that lets you mount individual files as if they were distinct devices.  It’s useful for mounting DVD-ROM images stored on disk or for developing filesystem images without having to worry about repartitioning.  Linux systems can even treat existing portions of the file tree as filesystems.  filesystems are attached to the tree with the mount command.  mount maps a directory within the existing file tree, called the mount point, to the root of the newly attached filesystem. 9  Filesystem Mounting and Unmounting (2)  The previous contents of the mount point become temporarily inaccessible as long as another filesystem is mounted there.  For example  installs the filesystem stored on the disk partition represented by /dev/sda4 under the path /users.  You could then use ls /users to see that filesystem’s contents.  You can run the mount command without any arguments  to see all the filesystems that are currently mounted. 10  Filesystem Mounting and Unmounting (3)  You detach filesystems with the umount command  umount complains if you try to unmount a filesystem that’s in use.  The filesystem to be detached must not have open files or processes whose current directories are located there and if the filesystem contains executable programs, none of them can be running.  Linux has a “lazy” unmount option (umount -l)  removes a filesystem from the naming hierarchy but does not truly unmount it until all existing file references have been closed.  umount -f  force-unmounts a busy filesystem and is supported on all our example systems. 11  Organization of the File Tree  UNIX systems have never been well organized.  Various incompatible naming conventions are used simultaneously, and  different types of files are scattered randomly around the namespace.  In many cases, files are divided by function and not by how likely they are to change,  making it difficult to upgrade the operating system.  The /etc directory, for example, contains some files that are never customized and some that are entirely local.  As a logically minded sysadmin, you may be tempted to improve the default organization.  Unfortunately, the file tree has many hidden dependencies, so such efforts usually end up creating problems.  Just let everything stay where the OS installation and the system packages put it. 12  Organization of the File Tree (1)  The root filesystem includes at least the root directory and a minimal set of files and subdirectories.  The file that contains the OS kernel usually lives under /boot,  but its exact name and location can vary.  Also part of the root filesystem are  /etc for critical system and configuration files,  /sbin and /bin for important utilities, and  sometimes /tmp for temporary files.  The /dev directory was traditionally part of the root filesystem,  but these days it’s a virtual filesystem that’s mounted separately 13 Organization of the File Tree (2) Standard directories and their contents 14  File Types  Most filesystem implementations define seven types of files.  Regular files  Directories  Character device files  Block device files  Local domain sockets  Named pipes (FIFOs)  Symbolic links  You can determine the type of an existing file with the file command.  know about the standard types of files  know a thing or two about common formats used within regular15 files.  File Types (1)  All that hoo-hah about /bin/sh means “it’s an executable command.”  Another option for investigating files is ls -ld  The -l flag shows detailed information  The -d flag forces ls to show the information for a directory rather than showing the directory’s contents.  The first character of the ls output encodes the type  For example, the circled d in the following output demonstrates that /usr/include is a directory: 16  File Types (2)  ls uses to represent the various types of files.  File-type encoding used by ls  rm is the universal tool for deleting files.  it’s a good idea to get in the habit of using rm’s -i option  to make rm confirm the deletion of each file.  ls -b shows control characters as octal numbers  To delete the most horribly named files, you might need to resort to rm 17 - i*  File Types (3)  Regular Files  Regular files consist of a series of bytes; filesystems impose no structure on their contents.  Text files, data files, executable programs, and shared libraries are all stored as regular files.  Both sequential access and random access are allowed.  Directories  A directory contains named references to other files.  You can create directories with mkdir and  delete them with rmdir if they are empty.  You can recursively delete nonempty directories - including all their contents—with rm -r.  The special entries “.” and “..” refer to the directory itself and to its parent directory; they cannot be removed.  Since the root directory has no real parent directory, the path “/..” is equivalent to the 18 path “/.” (and both are equivalent to /).  File Types (4)  Hard links  A file’s name is stored within its parent directory, not with the file itself.  more than one directory (or more than one entry in a single directory) can refer to a file at one time, and the references can have different names.  The filesystem maintains a count of the number of links that point to each file and does not release the file’s data blocks until its last link has been deleted.  You create hard links with ln and remove them with rm.  The command cp oldfile newfile creates a copy of oldfile called newfile, and ln oldfile newfile makes the name newfile an additional reference to oldfile.  In most filesystem implementations, it is technically possible to make hard links to directories as well as to flat files.  You can use ls -l to see how many links to a given file exist 19  File Types (5)  Character and block device files  Device files let programs communicate with the system’s hardware and peripherals.  Device drivers present a standard communication interface that looks like a regular file.  When the filesystem is given a request that refers to a character or block device file, it simply passes the request to the appropriate device driver.  Local domain sockets  Sockets are connections between processes that allow them to communicate hygienically.  Local domain sockets are accessible only from the local host and are referred to through a filesystem object rather than a network port. 20  File Types (5)  Named pipes  Like local domain sockets, named pipes allow communication between two processes running on the same host.  They’re also known as “FIFO files”  You can create named pipes with mknod and  remove them with rm.  Named pipes and local domain sockets serve similar purposes, and the fact that both exist is essentially a historical artifact.  Symbolic links  A symbolic or “soft” link points to a file by name.  The difference between hard links and symbolic links is that a hard link is a direct reference, whereas a symbolic link is a reference by name.  You create symbolic links with ln -s and remove them with rm  A symbolic link can contain either an absolute or a relative path. 21  File Attributes  Under Linux filesystem model  Every file has a set of nine permission bits  control who can read, write, and execute the contents of the file.  Together with three other bits  primarily affect the operation of executable programs, these bits constitute the file’s “mode.”  The twelve mode bits are stored along with four bits of file-type information.  The four file-type bits are set when the file is first created and cannot be changed  the file’s owner and the superuser  can modify the twelve mode bits with the chmod (change mode) command  Use ls -l (or ls -ld for a directory)  inspect the values of these bits. 22  The File Attributes (1) permission bits  Nine permission bits  determine what operations can be performed on a file and by whom.  Instead, three sets of permissions  define access for the owner of the file, the group owners of the file, and everyone else (in that order).  Each set has three bits:  a read bit, a write bit, and an execute bit (also in that order).  It’s convenient to discuss file permissions in terms of octal (base 8) numbers  because each digit of an octal number represents three bits  each group of permission bits consists of three bits.  The topmost three bits (with octal values of 400, 200, and 100) control access for the owner.  The second three (40, 20, and 10) control access for the group.  The last three (4, 2, and 1) control access for everyone else (“the world”). 23  File Attributes (2)  The permission bits (cont.)  In each triplet,  the high bit is the read bit, the middle bit is the write bit, and the low bit is the execute bit.  On a regular file,  the read bit allows the file to be opened and read.  The write bit allows the contents of the file to be modified or truncated;  however, the ability to delete or rename (or delete and then re-create!) the file is controlled by the permissions on its parent directory,  where the name-to-dataspace mapping is actually stored.  The execute bit allows the file to be executed. Two types of executable files exist:  binaries, which the CPU runs directly,  scripts, which must be interpreted by a shell or some other program. 24  File Attributes (2)  The permission bits (cont.)  For a directory,  The execute bit (often called the “search” or “scan” bit in this context)  allows the directory to be entered or passed through as a pathname is evaluated, but not to have its contents listed.  The combination of read and execute bits allows the contents of the directory to be listed.  The combination of write and execute bits allows files to be created, deleted, and renamed within the directory. 25  File Attributes (3)  setuid (Set user ID) and setgid (Set group ID) bits  The bits with octal values 4000 and 2000 are the setuid and setgid bits.  When set on executable files,  these bits allow programs to access files and processes that would otherwise be off- limits to the user that runs them.  When set on a directory,  the setgid bit causes newly created files within the directory to take on the group ownership of the directory rather than the default group of the user that created the file.  This convention makes it easier to share a directory of files among several users, as long as they belong to a common group.  This interpretation of the setgid bit is unrelated to its meaning when set on an executable file,  no ambiguity can exist as to which meaning is appropriate. 26  File Attributes (4)  The sticky bit  The bit with octal value 1000 is called the sticky bit.  If the sticky bit is set on a directory, the filesystem won’t allow you to delete or rename a file  unless you are the owner of the directory, the owner of the file, or the superuser.  The bits with octal values 4000 and 2000 are the setuid and setgid bits.  When set on executable files,  these bits allow programs to access files and processes that would otherwise be off- limits to the user that runs them.  Having write permission on the directory is not enough. 27  File Attributes (5)  ls: list and inspect files  The filesystem maintains about forty separate pieces of information for each file, but most of them are useful only to the filesystem itself.  As a system administrator, you will be concerned  mostly with the link count, owner, group, mode, size, last access time, last modification time, and type.  You can inspect all these with  ls -l (or ls -ld for a directory; without the -d flag, ls lists the directory’s contents).  An attribute change time is also maintained for each file.  The conventional name for this time (the “ctime,” short for “change time”) leads some people to believe that it is the file’s creation time  Unfortunately, it is not; it just records the time at which the attributes of the file (owner, mode, etc.) were last changed (as opposed to the time at which the file’s contents were 28 modified).  File Attributes (6)  The first field specifies  the file’s type and mode.  The first character is a dash,  so the file is a regular file.  The next nine characters in this field are  the three sets of permission bits.  The order is ownergroup-other, and the order of bits within each set is read-write- execute. Although these bits have only binary values,  ls shows them symbolically with the letters r, w, and x for read, write, and execute.  In this case,  the owner has all permissions on the file and everyone else has read and execute permission. 29  File Attributes (7)  ls: list and inspect files  One ls option that’s useful for scoping out hard links is -i,  tells ls to show each file’s “inode number.”  Briefly, the inode number is an integer associated with the contents of a file.  ls -li,  show link counts and inode numbers, and find, to search for matches.  Some other ls options that are important to know  -a : show all entries in a directory (even files whose names start with a dot),  -t : sort files by modification time (or -tr to sort in reverse chronological order),  -F : show the names of files in a way that distinguishes directories and executable files,  -R : list recursively, and -h to show file sizes in human-readable form (e.g., 8K or 53M). 30  File Attributes (8)  chmod: change permissions  The chmod (change mode) command changes the permissions on a file.  Only the owner of the file and the superuser can change a file’s permissions.  The first argument to chmod is a specification of the permissions to be assigned,  the second and subsequent arguments are names of files on which permissions should be changed.  In the octal case,  the first octal digit of the specification is for the owner,  the second is for the group,  and the third is for everyone else.  If you want to turn on the setuid, setgid, or sticky bits,  use four octal digits rather than three, with the three special bits forming the first digit. 31  File Attributes (8)  The eight possible combinations for each set of three bits, – where r, w, and x stand for read, write, and execute. – For example, chmod 711 myprog gives all permissions to the user (owner) and execute-only permission to everyone else. (If myprog were a shell script, it would need both read and execute permission turned on. 32  File Attributes (9)  Numeric Permissions – For example, chmod 711 myprog gives all permissions to the user (owner) and execute-only permission to everyone else. (If myprog were a shell script, it would need both read and execute permission turned on. 33  File Attributes (10)  For the mnemonic syntax, – you combine a set of targets (u, g, or o for user, group, other, or a for all three) with an operator (+, -, = to add, remove, or set) and a set of permissions. – Examples of chmod’s mnemonic syntax – The hard part about using the mnemonic syntax is remembering whether o stands for “owner” or “other”; “other” is correct. 34  File Attributes (11)  chown and chgrp: change ownership and group – The chown command changes a file’s ownership, – The chgrp command changes its group ownership.  The syntax of chown and chgrp mirrors that of chmod, – except that the first argument is the, respectively  To change a file’s group, you must either be the superuser or be the owner of the file and belong to the group you’re changing to. – Like chmod, chown and chgrp offer the recursive -R flag  to change the settings of a directory and all the files underneath it. – chown can change both the owner and group of a file at once with the syntax – You can actually omit either user or group,  which makes the chgrp command superfluous. – If you include the colon but name no specific group, 35  the Linux version of chown uses the user’s default group.  File Attributes (12)  umask: assign default permissions – You can use the built-in shell command umask  to influence the default permissions given to the files you create. – Every process has its own umask attribute;  The shell’s built-in umask command – sets the shell’s own umask, which is then inherited by commands that you run.  The umask is specified as a three-digit octal value that represents the permissions to take away. – For example, umask 027 allows all permissions for the owner but forbids write permission to the group and allows no permissions for anyone else. 36  File Attributes (13)  Linux bonus flags – Linux defines a set of supplemental flags that can be set on files to request special handling.  For example, the a flag makes a file append-only, and the i flag makes it immutable and undeletable. – Flags have binary values, so they are either present or absent for a given file. – Linux uses the commands lsattr and chattr  to view and change file attributes. 37  Access Control Lists  Access control lists – more powerful but also more complicated way of regulating access to files. – Each file or directory can have an associated ACL that lists the permission rules to be applied to it.  Each of the rules within an ACL is called an access control entry or ACE. – An access control entry identifies the user or group to which it applies and specifies a set of permissions to be applied to those entities. – The more sophisticated ACL systems let administrators specify partial sets of permissions or negative permissions. 38  Access Control Lists (1)  ACL types – Two types of ACLs have emerged as the predominant standards for Linux: POSIX ACLs and NFSv4 ACLs.  Systems have largely converged on a common framing for POSIX ACLs and a common command set, getfacl and setfacl, for manipulating them.  Implementation of ACLs – ACLs could be implemented  by the kernel on behalf of all the system’s filesystems,  by individual filesystems,  or perhaps by higher-level software such as NFS and SMB servers.  Linux ACL support – Linux has standardized on POSIX-style ACLs.  NFSv4 ACLs are not supported at the filesystem level, though of course Linux systems can mount and share NFSv4 filesystems 39  Working with Files  Checking file type – file command determines the file type  paul@laika:~$ file /etc/passwd /etc/passwd: ASCII text Creating an empty file – touch command create an empty file  paul@debian7:~$ touch file42 Removing a file – rm command remove a file that you no longer need  paul@debian7:~$ rm BigBattle.txt  rm -i To prevent yourself from accidentally removing a file  rm -rf will not remove non-empty directories 40  Working with Files (1)  Copy a file – copy one file  cp command to copy a file, use cp with a source and a target argument. – paul@debian7:~$ cp file42 file42.copy – copy to another directory  If the target is a directory, then the source files are copied to that target directory. – paul@debian7:~$ cp SinkoDeMayo dir42 – copy complete directories  cp -r (the -r option forces recursive copying of all files in all subdirectories). To copy complete directories – paul@debian7:~$ cp -r dir42/ dir33 – copy multiple files to directory  cp to copy multiple files into a directory. In this case, the last argument (a.k.a. the target) must be a directory. – paul@debian7:~$ cp file42 file42.copy SinkoDeMayo dir42/ 41  Working with Files (2)  Rename – rename files  mv command to rename a file or to move the file to another directory. – paul@debian7:~$ mv file42 file33 When you need to rename only one file then mv is the preferred command to use – rename directories  mv command can be used to rename directories – paul@debian7:~$ mv dir33 backup  The mv also has a -i switch similar to cp and rm. – mv -i will ask permission to overwrite an existing file. paul@debian7:~$ mv -i file33 SinkoDeMayo mv: overwrite `SinkoDeMayo'? no 42  Working with File contents  File contents – head  head command to display the first ten lines of a file – paul@debian7~$ head /etc/passwd  head command can also display the first n lines of a file. – paul@debian7~$ head -4 /etc/passwd  head can also display the first n bytes. – paul@debian7~$ head -c14 /etc/passwd – tail  tail command will display the last ten lines of a file. – paul@debian7:~$ tail /etc/services  You can give tail the number of lines you want to see paul@debian7:~$ tail -3 /etc/services 43  Working with File contents (1)  File contents – cat  Cat command to concatenate files into a bigger (or complete) file. – paul@debian8:~$ cat part1 part2 part3 >all – paul@debian8:~$ cat all one two three  cat command can also be used to create flat text files. – Type the cat > winter.txt command as shown in the screenshot below. Then type one or more lines, finishing each line with the enter key. After the last line, type and hold the Control (Ctrl) key and press d. paul@debian8:~$ cat > winter.txt It is very cold today! paul@debian8:~$ cat winter.txt It is very cold today! – The Ctrl d key combination will send an EOF (End of File) to the running process ending the cat command. 44  Working with File contents (2)  File contents – cat  cat command can be used to copy files. – paul@debian8:~$ cat winter.txt It is very cold today! – paul@debian8:~$ cat winter.txt > cold.txt – paul@debian8:~$ cat cold.txt It is very cold today! – tac  tac command is the opposite of cat (cat backwards) – Just one example will show you the purpose of tac (cat backwards) paul@debian8:~$ cat count one two three four paul@debian8:~$ tac count four three two 45 One