OS fundamentals - 2 - linux filesystem commands.pdf

OS FUNDAMENTALS Linux filesystem commands 1 Phaedra Degreef COMMAND LINE INTERFACE (CLI) SHELL COMMANDS • Users communicate with OS via a Command- Line Interface (CLI) − Using commands − Using a command shell • Set of commands • Commands can be different in different shells • Examples of she...

OS FUNDAMENTALS Linux filesystem commands 1 Phaedra Degreef COMMAND LINE INTERFACE (CLI) SHELL COMMANDS • Users communicate with OS via a Command- Line Interface (CLI) − Using commands − Using a command shell • Set of commands • Commands can be different in different shells • Examples of shells » » » » 3 bash sh ksh csh COMMAND OVERVIEW • Help command shows overview of available commands: 4 COMMAND PROMPT • Prompt: the system indicates that it is ready for a command Prompt (can be customized) 5 Cursor ANATOMY OF A COMMAND • 3 parts: command option(s) argument(s) • Example: • Use spaces between these parts!!!! 6 FILE SYSTEM WHAT IS A FILESYSTEM? • • • • 8 Organising files Apply security Virtualisation of storage hardware Provide API for file management FILESYSTEMS IN LINUX • ext2, ext3, ext4 • Tree structure − Directories (folders) − Files • File names: − Case-sensitive! − “unlimited” nr of characters − All characters except “/” (used in pathnames) allowed 9 FILESYSTEM HIERARCHY STANDARD (FHS) • Part of the LSB (Linux Standards Base) specifications − Which directories must be located on the first level and what they contain • Defines a two-layer hierarchy − Directories in the top layer (off the root “/”) − At the second layer under /usr and /var • Think of it as “what goes where and why” • Details: http://www.pathname.com/fhs/ 10 LINUX FILE SYSTEM STRUCTURE / bin sbin lib Root directory lib64 etc dev /bin • User binaries modules 11 − Essential binaries for all users (/bin) − Contains executables required when no other file systems are mounted. • For example, programs required for system booting, working with files and configuration. BASIC COMMANDS USED • cd: change directory • ls : list contents 12 Absolute path LINUX FILE SYSTEM STRUCTURE / bin sbin lib modules lib64 etc dev /sbin • System binaries − Contains programs important for system administration. − Typically are intended to be run by the root user and therefore it is not in the regular users path. 13 BASIC COMMANDS USED • pwd: show current directory • cd ../sbin − .. means “one level up” Relative path 14 RELATIVE PATH / ../sbin bin sbin current lib new modules 15 lib64 etc dev LINUX FILE SYSTEM STRUCTURE / bin sbin lib lib64 etc dev • Libraries modules 16 − Many programs have common functions they need. The functions can be kept in a shared library. Libraries are called shared objects and end with the .so extension. − Libraries in /lib are used by programs in /bin and /sbin. − There are additional libraries in subdirectories. Kernel modules are located in /lib/modules LINUX FILE SYSTEM STRUCTURE / bin sbin lib modules 17 lib64 etc dev • /etc : Configuration files − /etc contains system and services configuration files. Most of these files are ASCII files. − Normal users can read most of these by default. This can be a security issue since some of these files contain passwords so it important that these files are only readable by the root user. − No executables can be put here according to the FHS, however subdirectories may contain shell scripts. − Almost every installed service has at least one configuration file in /etc or a subdirectory. LINUX FILE SYSTEM STRUCTURE / etc dev usr opt • /usr : Unix System Resources bin sbin lib include share man 18 doc − /usr/bin/ - Almost all executables − /usr/lib/ - Libraries and application directories − /usr/sbin/ - System administration programs − /usr/share/doc/ - Documentation − /usr/share/man/ - Manual pages LINUX COMMANDS CLI • Command-Line Interface • Why? − Perfect for automation of (system management or other) actions • Via scripts » Collection of CLI commands and options • Part of a “shell” − A collection of commands − Available shells are listed in /etc/shells 20 EXAMPLE • Try to copy all files with filename − File extension .dll − Name ending with the letter “r” Copy these files to C:\tmp • Try this using the Windows Graphical User Interface (GUI) • 21 COPY USING THE COMMAND-LINE 22 NOW TRY THIS • 23 Try to copy the same files, but also from all subdirectories that contain the number 3 in their name. HOW TO GET HELP • Command help: command -h or command -• • • • • help Manual pages: man [section] command List for keywords: man -k keyword Display descriptions: whatis command Info pages: info command Software package documentation in − /usr/share/doc/packages/ • Release Notes in /usr/share/doc/release- notes/ 24 HOW TO USE THE HELP TOOLS • man man ☺ 25 NAVIGATING HELP (INFO, MAN) • Info pages have a tree structure with nodes and references between the nodes. In the top line, you can see the name of the node, the node name of the next page, the name of the previous node and the name of the parent node 26 NAVIGATING HELP (INFO, MAN) • • • • • • • • • • • • • • • • 27 q Quit n Next node p Previous node u Up to parent node l Last node visited b Beginning of node e End of node t Top of node Enter Follow reference Down One line down Up One line up Space, PageDown Page down one screen Backspace, PageUp Page up one screen Tab Jump to the next reference s Search in the info page ? List a summary of commands SHELL HISTORY • Command history in ~/.bash_history • View history: history • Scroll through history − <Up-arrow> key / <Down-arrow> key • Search history for a command that begins with <string> − <string><PgUp> / <string><PgDn> • Search history for any <string>, press Ctrl+R − Execute: Press Enter − Change: Press Tab • Re-execute a numbered command from history − !<command-number> 28 AUTO-COMPLETION 29 WILDCARDS • Occasionally, you might want to perform operations on a series of files without having to name all the files. The Bash shell uses special tokens, commonly referred to as “wildcards”, to reference multiple values. 30 COMMON CLI COMMANDS • Navigate the Filesystem (cd/pwd/...) • Manipulate Files and Directories • • • • • 31 (mkdir/rmdir/rm/cp/rsync/dd/...) Display File Contents (cat/less/tail/head/...) Search the Filesystem (find/locate/...) Search and Alter File Contents (grep/egrep/sed/awk...) Process Text (cut/sort/wc/...) Archive Files and Directories (tar/gzip/...) COMMON SHELL COMMANDS: FILE SYSTEM 32 • There are some common “abbreviations” for special locations in the file system: . – represents the current directory .. – represents the parent directory ~ – represents the home directory ~username – represents the home directory of user username 33 LS COMMAND • The ls (list short) command lists specified files. If a directory is included with ls, the directory's contents are displayed. Without an option, the contents of the current directory are listed. • The most important options of ls: • Options can be combined: ls -al 34 FILETYPES • • • • 35 Regular files Directories Symbolic links Others…. SYMBOLIC LINKS • Hard link (default) − Command: ln <filename> <linkname> • Symbolic link (soft link) − Command: ln –s <filename> <linkname> • inode = file metadata − − − − 36 Ownership Security Physical location … LINKS • Hard links can only be used when both the file and the link are in the same file system (on the same partition), because inode numbers are only unique within the same file system. • A symbolic link is assigned its own inode—the link refers to a file, so a distinction can always be made between the link and the actual file. 37 • If you enter pwd with the -P option, pwd prints the physical directory without any symbolic links: geeko@server1:~ > ls -l doc/ lrwxrwxrwx 1 geeko users 15 2004-02-12 08:43 doc -> /usr/share/doc/ geeko@server1:~ > cd doc/ geeko@server1:~ > pwd /home/geeko/doc geeko@server1:~ > pwd -P /usr/share/doc geeko@server1:~ > 38 TOUCH COMMAND • You can use the touch command to change the time stamp of a file or to create a new file with a size of 0 bytes. • The following are the most important options: Option Description -a Changes only the time of the last read access (access time) -m Changes only the time of the last modification (modification time). -r file Sets the time stamp of file instead of the current time -t time Instead of the current time, sets time (structure: [[CC]YY]MMDDhhmm.[ss] ([Century]Year] Month Day Hour Minute [Seconds], two digits in each case)) 39 EXAMPLE 40 AANDDD…SOME MORE COMMANDS • Copy command : cp − cp source_file dest_file − cp file1 file2 file3 dest_directory − cp –r source_dir dest_dir • rsync: synchronising 2 directories on different machines 41 MOVE AND REMOVE • Move command : mv − mv old_file new_file • Delete (remove) command: rm − rm filename_tobedeleted − rm –r directoryname : recursive delete • Make new directory: mkdir • Remove directory: rmdir 42 “ESCAPING” • To prevent the shell from interpreting special characters in the command line, these characters must be “masked” 43 EXAMPLES 44 DISPLAY FILE CONTENT 45 CAT, TAC 46 LESS & MORE COMMAND • You can use the less command to display the contents of a file page by page. Even compressed files (such as .gz and .bz2) can be displayed. • You can use the following keystrokes with less: • more works similar to less, but is more limited in its functionality. more quits if you reach the end of a file automatically, whereas less does not. 47 HEAD, TAIL COMMAND 48 TAIL -F • When used with the tail command, the -f option displays a continuously updated view of the last lines of a file. • If a line is added at the end of the file while tail -f is running, the line is displayed. • This is a very useful feature for observing log files. • To exit tail -f, press Ctrl+C 49 GREP • The grep command and its variant egrep are used to search files for certain patterns using the syntax grep [options] search_pattern filename • The command searches filename for all text that matches search_pattern, and prints the lines that contains the pattern. 50 REGULAR EXPRESSIONS • Regular expressions are strings consisting of meta characters and regular characters and numerals (also known as “literals”). • In the context of regular expressions, metacharacters are those characters that do not represent themselves but have special meanings. − They can act as placeholders for other characters or can be used to indicate a position in a string. 51 REGULAR EXPRESSIONS • You can specify search patterns in the form of regular expressions, although the basic grep command is limited in this regard. To search for more complex patterns, use the egrep command (or grep -E) instead, which accepts extended regular expressions. 52 REGULAR EXPRESSIONS 53 DISPLAY FILE CONTENT • • • • • • • • 54 cat <filename> less <filename> more <filename> head –n 10 <filename> tail –n 10 <filename> grep <filename> texttosearch grep –r * texttosearch sort <filename> FIND COMMAND • To search for files on the command line, you can use the find command. The following is the syntax for the find command: find path criterion [action] − path. The section of the file system to search (the specified directory and all its subdirectories). If nothing is specified, the file system below the current directory is used. − criterion. The properties the file should have − action: Options that influence the following conditions or control the search as a whole 55 FIND TYPICAL EXAMPLE 56 FIND, MORE ELABORATE EXAMPLE geeko@server1:~> find ~ -name “letter*” -type f -exec grep appointment {} \; appointment for next meeting: 23.08. /home/geeko/letters/letter_Smith • In this example, the find command searches for files whose names begin with the word “letter”, and then passes the names of the files found with -exec to the following command (in this case, grep appointment {}). • The two brackets “{}” stand as placeholders for the file names which are found and passed to the grep command. • The semicolon closes the -exec instruction. Because this is a special character, it is masked by placing a backslash in front of it. 57 DATA CHANNELS • Input and output to processes in Linux can be referred to as "data channels". 58 − The data channel from which a process receives input is called standard in or "stdin" and has a file descriptor number of 0. − The data channel to which a process sends output is called standard out or "stdout" and has a file descriptor number of 1. − Error messages are split out from standard output and sent to their own channel called standard error or "stderr", which has a file descriptor number of 2. CHANNEL REDIRECTION 59 CHANNEL REDIRECTION EXAMPLES 60 CHANNEL REDIRECTION EXAMPLES • Our previous find command gave error output: • With redirection of error output: 61 CHANNEL REDIRECTION EXAMPLES 62 PIPES 63 TEE 64 CHAINING COMMANDS 65 PIPES • Redirecting – next level • Redirect the redirected cat *.log | grep –i error | sort • Tee option: t-join make | sudo tee build.log 66 COMMAND SUBSTITUTION • Sort of pipes • Command1 <options> $(command2 <options>) − Executes command2 − Sends output of command 2 to command1 • du –h –a –c $(find . –name *.conf 2>&-) − Shows the size of all the files found by the find command − 2>&- discards all possible error output from the find command 67 ALIASES • Aliases allow to “rename” commands − Easier to remember or easier to type • They are defined in the file named .bashrc in every users’ home directory (in Ubuntu Linux) − Other Linux distributions store them in a separate .alias file ll is an alias for ls –al 68 ALIASES • When we look into the .bashrc file: • Even the ls command has an alias (commented out here, because I’m not a fan of the colors used) − (change to ls –color=none if you don’t want the colors) Example of colors used: 69 ADDING ALIASES • Use the alias command in the command line − They will only be used in the current session, and will be removed when you log out • Add your own aliases to the .bashrc file − They will automatically be added when you log in. 70 BACKUP-ORIENTED COMMANDS 71 SPECIAL FILE-COPYING COMMANDS • Instead of creating an archive file, rsync creates a mirror copy of the data being backed up in the file system of the destination device. 72