UNIX_Module-1_Final.pdf

History of Unix In 1969 Kenneth Thompson and Dennis Ritchie developed the UNIX operating system at Bell Labs. Unix is the first operating system designed to run on computers of all sizes, making open systems possible. UNIX became the foundation for the Internet....

History of Unix In 1969 Kenneth Thompson and Dennis Ritchie developed the UNIX operating system at Bell Labs. Unix is the first operating system designed to run on computers of all sizes, making open systems possible. UNIX became the foundation for the Internet. 5  History of Unix "Until UNIX came on the scene, operating systems were designed with a particular machine in mind. They were written in a low-level language (like assembler, which uses human unreadable code). The systems were fast, but were restricted to the hardware they were designed for. Programs designed for one system would not run on the other. This was the status when Ken Thompson and Dennis Ritchie, of AT&T fame, authored the UNIX system. 6  History of Unix "The origins of Unix date back to the mid-1960s when the Massachusetts Institute of Technology (MIT), Bell Labs, and General Electric were developing Multics, a time- sharing operating system for the GE-645 mainframe computer. Multics featured several innovations, but also presented severe problems: size and complexity, individual researchers at Bell Labs started withdrawing from the project. The last to leave were Ken Thompson, Dennis Ritchie, Douglas McIlroy, and Joe Ossanna, who decided to reimplement their experiences in a new project of smaller scale. The new operating system was a single-tasking system, which was initially without organizational backing, and also without a name. 6  History of Unix In 1970, the group coined the name Unics for Uniplexed Information and Computing Service ("eunuchs"), Later Brian Kernighan takes credit for the idea, but adds that "no one can remember" the origin of the final spelling Unix. Dennis Ritchie, Doug McIlroy, and Peter G. Neumann also credit Kernighan. The operating system was originally written in assembly language, but in 1973, Version 4 Unix was rewritten in C. Version 4 Unix, however, still had many PDP-11 dependent codes, and was not suitable for porting. 7  History of Unix 1978 the first port to another platform was made to for the Interdata 8/32O. "Bell Labs produced several versions of Unix that are collectively referred to as "Research Unix". In 1975, the first source license for UNIX was sold to Donald B. Gillies at the University of Illinois at Urbana–Champaign Department of Computer Science. UIUC graduate student Greg Chesson, who had worked on the UNIX kernel at Bell Labs, was instrumental in negotiating the terms of the license. During the late 1970s and early 1980s, the influence of Unix in academic circles led to large- scale adoption of Unix (BSDO and System V) by commercial startups including DYNIX, HP-UX, SunOS/Solaris, AIX, and Xenix. 8  History of Unix In the late 1980s, AT&T Unix System Laboratories and Sun Microsystems developed System V Release 4 (SVR4), which was subsequently adopted by many commercial Unix vendors. In the 1990s, Unix and Unix-like systems grew in popularity and became the operating system of choice for over 90% of the world's top 500 fastest supercomputers, as BSD and Linux distributions were developed through collaboration by a worldwide network of programmers. In 2000, Apple released Darwin, also a Unix system, which became the core of the Mac OS X operating system, later renamed macOS. 2020 Unix OS’s are widely used in modern servers, workstations, and mobile devices. 9 Meanwhile Microsoft was making it big with windows – GUI –Windows 3.1/95/98 – for desktops –Windows NT/2000 – for servers. So UNIX needed windows type interface for survival and introduced X windows @ MIT. Finally UNIX turned commercial. –Richard Stallman – Started Free software foundation GNU –Linus Torvalds – Linux OS Many Linux tools are written and supplied free by GNU(GNU's Not Unix”). UNIX is a giant operating system and command oriented  The history of Linux is quite fascinating! It all began in 1991 when Linus Torvalds, a Finnish computer science student, started a personal project to create a free operating system kernel. He was inspired by MINIX, a UNIX- like operating system Today, Linux is one of the most widely used operating systems in the world, powering everything from smartphones (like Android) to supercomputers. Linux’s open-source nature has allowed it to evolve rapidly, with contributions from thousands of developers globally. It’s known for its stability , security, and flexibility, making it a popular choice for a wide range of applications Here are some key milestones in the history of Linux: 1991: Linus Torvalds released the first version of the Linux kernel. It was initially a small project, but it quickly gained attention from developers around the world. 1992: Linux was re-licensed under the GNU General Public License (GPL), which allowed anyone to use, modify, and distribute the software freely. 1994: The first stable version of the Linux kernel, version 1.0, was released. 1996: The Linux mascot, Tux the penguin, was chosen. 2000s: Linux began to be widely adopted in various fields, including servers, supercomputers, and embedded systems.  Here’s a brief overview of UNIX, Linux, BSD, and Solaris: UNIX Origin: Developed in the late 1960s at AT&T’s Bell Labs by Ken Thompson and Dennis Ritchie. Features: Multi-user, multi-tasking capabilities, hierarchical file system, and modularity. Influence: The foundation for many other operating systems, including Linux and BSD Linux Origin: Created by Linus Torvalds in 1991 as a free and open-source alternative to UNIX. Features: Highly customizable, supports a wide range of hardware, and has a large community of developers. Distributions: Includes popular distributions like Ubuntu, Fedora, and CentOS  BSD (Berkeley Software Distribution) Origin: Emerged from UNIX in the late 1970s at the University of California, Berkeley. Features: Known for its performance, security, and networking capabilities. Variants: Includes FreeBSD, OpenBSD, and NetBSD, each with its own focus and strengths Solaris Origin: Developed by Sun Microsystems in 1992, now maintained by Oracle. Features: Known for its scalability, advanced features, and the ZFS file system. Use Cases: Often used in enterprise environments for its robustness and support for large-scale applications Outline Introduction of UNIX Introduction to OS Architecture Experience the Unix environment Basic commands  Introduction to OS Operating System (OS) It is a software that manages the computers hardware and provides a convenient and safe environment for running programs. It acts as an interface between programs and the hardware resources that these programs access (like memory, hard disk and printers). It is loaded into memory when a computer is booted and remains active as long as the machine is up. OS Architecture Figure: Layout of Operating System Introduction What is UNIX? UNIX operating system is a set of programs that plays a role as a connection between the computer and the user. Unix is an Operating System which is the base of all Operating Systems like Ubuntu, Solaris, POSIX, etc. It was developed in the 1970s by Ken Thompson, Dennis Ritchie, and others in the AT&T Laboratories. Originally it was written in assembler, later rewritten in C. Users communicate with the kernel(part of OS) through a program known as the shell. A shell is a command line interpreter. A shell is a program that acts as the interface between the user and the UNIX system, allowing the user to enter commands for the operating system to execute.  Kernel file LINUX: /boot/vmlinuz Shell: Bash Shell by default C (csh) shell and Korn (ksh) Shell also offered. echo $SHELL : Displays the shell running Two entities: File and Process File has place Process has life System Call: handful functions to communicate with the kernel Example: read(), write(), open(), close() etc. Figure: The Kernel-Shell Relationship 1. Division of labor Kernel Core of the operating system Collection of routines written in C It is loaded into memory when the system is booted and communicates directly with hardware. User program uses the services of the kernel to access the hardware using the system calls Kernel Manages system memory, schedules process, decides priorities and performs other tasks which you don’t even bother. The kernel has to do lot of work even when if no user program is running. It is often the operating system. Shell Role: The shell acts as a command interpreter. It translates user commands into actions that the kernel can execute. Though only one kernel is running on the system, there could be several shells in action for each user who is logged in. $echo $SHELL Types: Sh (Bourne Shell) ($): One of the earliest UNIX shells, known for its simplicity and scripting capabilities. Developed by Steve Bourne at AT&T Labs Because it is oldest and primitive it is not used in most systems today An enhanced version of Bourne Shell called Bash (Bourne again shell) is used in Linux.  Csh (C Shell) (%): Developed by Bill Joy in Berkeley Compatible version of C shell is tcsh used in Linux. Introduced features like command history and scripting syntax similar to the C programming language. Received a name from the fact that its commands look like C statements Ksh (Korn Shell): Combines features of both Bourne and C shells, offering enhanced scripting capabilities and performance. Developed by David Korn at AT&T Labs It is the newest and most powerful It is compatible with Bourne shell The default system prompt for the bourne, bash and korn shells is dollar ($).  The file and process Two simple entities support the UNIX system – the file and process. File: Definition: A file is an array of bytes that can contain virtually anything, such as text, images, or executable code. Process: Definition: A process is the name given to a file when it is executed as a program. It represents a running instance of a program. Files have places and processes have life.  File File is just array of bytes and contain virtually anything. It is related to another file by being part of a single hierarchical structure. The dominant file type is text Unix provides a vast array of text manipulation tools that can edit these files without using an editor. Process Is the name given to file when is executed Unix provides tools that allow us to control processes, move them between foreground and background.  The System Calls System calls are the interface between user applications and the kernel. They allow programs to request services from the operating system. Examples: Common system calls include open, write, read, and close. POSIX (Portable Operating System Interface) Definition: POSIX is a set of standards that ensure compatibility between different UNIX-like operating systems. It defines a consistent interface for system calls and other operating system services. Unix system- comprising the kernel, shell and applications is written in C The commands use the functions called system call to communicate to the kernel Unix command writes the file using write system call: read(), write(), open(), close() Same system call can access both file and device  Services Windows API calls UNIX system calls CreateProcess() Fork() Process Control ExitProcess() Exit() WaitForSingleObject() Wait() Open() CreateFile() Read() File manipulation ReadFile() Write() WriteFile() Close() SetConsoleMode() Ioctl() Device Management ReadConsole() Read() WriteConsole() Write() GetCurrentProcessID() Getpid() Information Maintenance SetTimer() alarm() Sleep() Sleep() CreatePipe() Pipe() Communication CreateFileMapping() Shmget() MapViewOfFile() Mmap() SetFileSecurity() Chmod() Protection InitializeSecurityDescriptor() Umask() SetSecurityDescriptorgroup() Chown() Examples of system calls in windows and UNIX Unix Architecture Figure: Unix architecture Features of UNIX 1. UNIX: A Multiuser system 2. UNIX: A Multitasking system 3. The Building Block Approach 4. The UNIX toolkit 5. Pattern Matching 6. Programming Facility 7. Documentation UNIX Features Multiuser support: UNIX is a Multiuser system, i.e. UNIX allows multiple users to simultaneously access the same system and share resources such as CPU, memory, and disk space. This is managed efficiently to ensure that each user gets a fair share of the resources. Multitasking: Unix is a multitasking operating system. This means it can handle multiple tasks or processes at the same time. 1.Unix uses a process scheduler to manage the execution of multiple processes. Each process is given a fair share of CPU time, allowing for efficient multitasking. 2.Unix employs a time-sharing mechanism where the CPU time is divided among all active processes. This ensures that each process gets a chance to execute, making the system responsive. 3.Background and Foreground Processes: Users can run processes in the background, allowing them to continue working on other tasks in the foreground. This is particularly useful for long-running tasks.  Building-block approach The building-block approach in Unix is a fundamental design philosophy that emphasizes simplicity and modularity. Unix provides a large number of small, simple commands, each designed to perform a specific task. This makes it easier to understand and use each command individually. Unix allows you to connect these small commands using pipes (|). A pipe takes the output of one command and uses it as the input for another. This enables the creation of complex workflows by chaining simple commands together. For example, ls | wc lists the files in a directory and then counts them Modularity: By combining small, modular commands, users can accomplish complex tasks. Reusability: The building-block approach encourages reusability. Since each command is designed to do one thing well, you can use the same commands in different combinations to solve various problems. Efficiency: This approach also promotes efficiency. Instead of writing large, monolithic programs, you can use existing commands to build new functionality quickly and efficiently  UNIX Toolkit: The Unix operating system introduced the idea of a toolkit. The UNIX toolkit refers to a collection of small, powerful, and flexible command-line utilities that can be combined to perform complex tasks and to create custom programs. General purpose tools, text manipulation utilities, compilers and interpreters, networked applications, System administration tools etc. Because Unix treats files, commands, and devices all pretty much as files, and since most of these tools will take their input either from files or from other commands (or standard input if no file name is given), this combination of tools is general and powerful.  Pattern matching : Pattern matching in UNIX is a powerful feature used to search and manipulate text based on specific patterns. It often involves the use of metacharacters and regular expressions Metacharacter : Metacharacters are special characters that have specific meanings in pattern matching... Matches any single character except a newline. Example: a.b matches aab, acb, etc. * Matches zero or more occurrences of the preceding element. Example: a* matches a, aa, aaa, etc. Regular expression : They allow you to define patterns to match specific sequences of characters within text files or streams. Example: [a-zA-Z]\d{2}  Programming facility –Unix shell is a programming language. –It has control structures, loops and variables and are used in shell scripts-programs that can invoke the UNIX commands. They help automate repetitive tasks, by executing a series of commands in sequence, making them more efficient and less error-prone  Documentation In Unix, documentation is crucial for understanding and effectively using the system. 1. Man Pages: Man pages (short for manual pages) are the primary source of documentation in Unix. They provide detailed information about commands, system calls, library functions, and more. To access a man page, use the man command followed by the name of the command or topic. For example: man ls. This command will display the manual page for the ls command 2. Info Pages: Info pages are another form of documentation, often more detailed than man pages. They are accessed using the info command. For example: info ls This command will display the info page for the ls command 3. Help command: Many Unix commands have a built-in help option that provides a brief overview of their usage. For example: ls --help This command will display a summary of the ls command’s options and usage. Types of Unix Operating Systems Unix gives the user the flexibility to modify the Unix code according to their requirements. Thus it is easy to port the Unix code on different types of hardware. The freely available source code of this operating system and its portability feature lead to various Unix like operating systems. Some of the popular Unix like operating systems are Solaris, Darwin, AIX, HP-UX, FreeBSD, NetBSD, Xenix, IRIX, Tru64, macOS, etc… The UNIX Environment Unix is a multiuser, multiprocessing, portable operating system designed to facilitate programming, text processing, communication and many other tasks that are expected from an operating system. It is used in three different computing environments 1. Stand alone personal environment 2. Time sharing systems 3. Client server system Stand alone personal environment Personal Computing Environment : In personal computing environment there is a stand-alone machine. Complete program resides on computer and executed there. Different stand-alone machines that constitute a personal computing environment are laptops, mobiles, printers, computer systems, scanners etc. Unix is originally designed as a multiuser environment, many users were installing Unix on their personal computers. This trend made up with the availability of Linux, a free Unix system  Time-Sharing Environment Time-sharing allows multiple users to share the computing resources of a single machine simultaneously. The operating system allocates time slices to each user, giving the impression of concurrent execution.  Client/Server Environment In a client/server environment, multiple client machines request services and resources from a central server. The server processes these requests and returns the results to the clients. UNIX structure The kernel: Process control, Resource management The shell: interpreter, programming capability Utilities: Text editors, search programs, sort , Application 1. The Kernel The kernel is the core of the UNIX operating system. It manages the system's resources and allows other programs to run and use these resources. Key functions include: - Process Control: Manages the execution of processes, including multitasking, process scheduling, and inter-process communication. - Resource Management: Handles memory allocation, file system management, and device control. 2. The Shell The shell acts as an interface between the user and the kernel. It interprets user commands and executes them. It also provides programming capabilities through scripting. - Interpreter: Translates user commands into actions performed by the kernel. - Programming Capability: Allows users to write shell scripts to automate tasks. 3. Utilities UNIX comes with a variety of utility programs that perform specific tasks. These include: - Text Editors: Programs like `vi` and `nano` for editing text files. - Search Programs: Tools like `grep` for searching text using patterns. - Sort Programs: Utilities like `sort` for arranging text data. 4. Applications Applications are user-level programs that perform specific tasks. These can range from simple command-line tools to complex software suites. It is not a part of standard Unix but is written by system administrators, professional programmers, or users to extend the capability to the system POSIX and the Single UNIX Specification Dennis Ritchie's decision to rewrite UNIX in C didn’t quite make UNIX very portable. UNIX fragmentation and the absence of a single conforming standard adversely affected the development of portable applications. System V Interface Definition (SVID): Created by AT&T, this was an attempt to standardize the UNIX System V operating system, making it easier for developers to write applications that could run on any System V-compliant system. X/Open Portability Guide (XPG): Developed by X/Open (now The Open Group), this guide aimed to create a more unified standard for UNIX systems. Products conforming to this specification were branded as UNIX95, UNIX98, or UNIX03, indicating their compliance with the respective versions of the XPG.  POSIX and the Single UNIX The Portable Operating System Interface (POSIX) standards, developed by the IEEE, Specification were indeed a major step towards enhancing the portability and interoperability of UNIX-based systems. POSIX Standards: These standards were designed to ensure compatibility and portability across different operating systems, with a strong foundation in UNIX. POSIX.1: This part of the standard specifies the C application programming interface (API), including system calls, which are essential for developing portable applications. POSIX.2: This part deals with the shell and utilities, providing a standard environment for scripting and command-line operations.  POSIX and the Single UNIX Specification Single UNIX Specification, Version 3 (SUSV3): In 2001, the unification of POSIX and the X/Open standards resulted in SUSV3, which further streamlined and standardized UNIX systems. The “Write once, adopt everywhere” philosophy is central to POSIX, allowing software developed on one POSIX-compliant system to be easily ported to another with minimal modifications. This approach has been instrumental in making UNIX a versatile and widely adopted operating system. General features of Unix commands/command structure. Command arguments and options. Command Structure Since the introduction of UNIX System V, Release 3, any new commands must obey a particular syntax governed by the following rules: Command names must be between 2 and 9 characters in length. Command names must be comprised of lowercase characters and digits. Option names must be one character in length.  command[options] [arguments] where an argument indicates on what the command is to perform its action, usually a file or series of files.... Commands are case sensitive.... Options are generally preceded by a hyphen (-), and for most commands, more than one option can be strung together, in the form: ls -alR will perform a long list on all files in the current directory and recursively perform the list through all sub-directories. For most commands you can separate the options, preceding each with a hyphen, e.g.: Command -option1 -option2 -option3 as in: Ls -a -l -R  Command structure The first word is the command The additional words are arguments Ex: $cat abc.txt Commands and arguments must be separated by spaces or tabs to enable system to interpret them as words. Any number of spaces can be used. A contiguous string of spaces and tabs is called whitespace. Shell compresses multiple consecutive spaces or tabs to a single space. Unix arguments consists of options, expressions, instructions, filenames, etc.  Example: ls -l /home/user ls: Lists directory contents. -l: Option to use a long listing format. /home/user: Argument specifying the directory to list.  Command arguments and options A special type of argument mostly used with (-) minus sign. Ex: $ls -a Options are normally preceded by a - sign to distinguish them from filenames. Options list is predetermined. Command [Options] [Arguments] Wrong option given- the shell locates the command but command finds the wrong option. Ex: $ls –z The error message is generated by the command and not the shell. Options are combined with only one (-) sign. Ex: $wc -c -w -l abc.txt or $wc -cwl abc.txt  Whitespace Cat README Options ls -l ls -l filename ls –l –a –t ls –lat Command structure 2. Filename Arguments ls –lat chap01 chap02 chap03 cp chap01 chap02 rm chap01 chap02 3. Exceptions who ls pwd etc…. Meaning of Internal and external commands Meaning of Internal and external commands Internal commands are commands that are already loaded in the system. They can be executed any time and are independent. Example: cd, source, XCOPY, CHKDSK, LABEL, TREE etc External commands are loaded when the user requests for them. Example: ls, cat etc Internal and External Commands Internal Commands These commands are built into the shell itself. They are executed directly by the shell without needing an external executable file. Examples: cd, echo, pwd, exit. Generally faster because the shell doesn’t need to search for the command in the file system. External Commands These commands are not built into the shell. They exist as separate executable files in the file system. The shell locates these commands in directories specified by the PATH variable and then executes them. Examples: ls, cat, grep, find. Slightly slower compared to internal commands because the shell needs to find and execute the corresponding file. If a command has entry in built-in set and has external file also, in such situation internal command takes precedence. The type command: knowing the type of a command and locating it. The type command in Linux is used to determine how a given command will be interpreted by the shell. It helps you understand whether a command is an alias, a shell function, a built-in command, a disk file, or a reserved word. type [OPTIONS] COMMAND_NAME Common Outputs: alias: If the command is an alias (a shortcut to another command). builtin: If the command is a built-in function of the shell itself. file: If the command is an executable file. function: If the command is a shell function. keyword: If the command is a reserved word in the shell. Options -t: Print a single word indicating the type of command. -a: Display all locations that contain the command. -p: Return the path to the command if it is an executable file. -P: Search the PATH for an executable file even if the command is not a file. -f: Do not look up shell functions.  type ls Output might be: ls is aliased to `ls --color=auto` Display the type of multiple commands: type pwd date Output might be: pwd is a shell builtin date is /usr/bin/date Use the -t option to get a terse output: type -t echo Output might be: builtin Show all locations containing the command with the -a option: type -a pwd Output might be: pwd is a shell builtin pwd is /bin/pwd type -a cd Output might be: cd is a shell builtin Understanding of some basic commands [echo, printf, ls, who, date, passwd, cal, Combining commands.] echo: Display message Display diagnostic messages on the terminal, or issue prompts for taking user input To display message (echo Sun Solaris) To evaluate shell variables (echo $SHELL) Echo behavior differs across the shells, and most of these differences are related to the way the echo interprets the escape sequence The echo command is a versatile and commonly used command in various operating systems, including Linux and Windows. It is primarily used to display text or the results of commands to the terminal or console. Print a simple string: echo “Hello, World!” Suppress the newline at the end of the output: echo -n “Hello, World!” Hello, World!user@SJEC:$ To use these escape sequences, you need to use the -e option with the echo command. Escape Sequence Description \a Alert (bell). Produces an audible or visible alert, depending on the system. \b Backspace. Moves the cursor one position to the left. \c Suppress further output. Echo will not output anything else that follows. \e Escape character. \f Form feed. Moves the cursor to the start of the next page. \n New line. Moves the cursor to the next line. \r Carriage return. Moves the cursor to the beginning of the line. \t Horizontal tab. Adds a tab space. \v Vertical tab. Moves the cursor down to the next vertical tab stop. \\ Backslash. Prints a backslash character. \' Single quote. Prints a single quote character. \" Double quote. Prints a double quote character. Enable interpretation of backslash escapes: echo -e “Hello,\nWorld!” Hello, World! echo -e “Hello,\t World!” Hello, World! Using variables: name="Alice" echo "Hello, $name!“ Redirecting output to a file: echo "Hello, World!" > output.txt echo $ echo UNIX $ echo $SHELL $ echo $x $ x=3 $ echo $x printf: An alternative to echo The echo and printf commands are both used to display text, but they have some key differences. Like echo printf exists as an external command but only the Bash shell has printf built-in. $printf "No filename entered\n" No filename entered printf does not automatically insert a newline unless the \n is used explicitly printf also uses formatted strings in the same way the C language function $ printf "My current shell is %s\n" $SHELL My current shell is /bin/bash printf allows for detailed formatting, including specifying width, precision, and alignment, making it suitable for more complex output needs. echo is straightforward and does not support advanced formatting. echo is generally more portable across different Unix-like systems. printf can behave differently on various systems, especially with different versions of the shell  Escape Sequences Format Strings \a Bell %s string \b Backspace %30s printed in space 30 characters wide \c No newline %d decimal integer \f Formfeed %6d printed in space 6 characters wide \n Newline %f floating point number \r Carriage return \t Tab %o octal \v Vertical tab %x hexadecimal \\ Backslash \0n ASCII characters represented octal value printf "My name is %s and USN is %d\n" $name $number My name is Alice and USN is 12345 num=10 $ printf "My current shell is %x\n" $num My current shell is a printf Similar to echo Differ in newline ls: listing files in current directory The ls command in UNIX is used to list files and directories within the file system 1. Basic Listing Command: $ ls Description: Lists files and directories in the current directory. $ ls /home/user Description: Lists files and directories in the /home/user directory The files are arranged alphabetically with uppercase having precedence over lower. Common Options: -l: Long listing format. Displays detailed information about each file, including permissions, number of links, owner, group, size, and timestamp. -a: Lists all files, including hidden files (those starting with a dot). -h: Human-readable format. Makes file sizes easier to read (e.g., 1K, 234M, 2G). -R: Recursively lists subdirectories. -t: Sorts files by modification time, newest first. -S: Sorts files by size, largest first. Long Listing Format Command: ls -l Description: Provides detailed information about each file and directory. Example Output: -rw-r--r-- 1 user group 1234 Aug 12 20:00 file.txt drwxr-xr-x 2 user group 4096 Aug 12 20:00 directory Show Hidden Files Command: ls -a Description: Lists all files, including hidden files (those starting with a dot). Example Output:.bashrc.profile file.txt directory  Human-Readable Sizes Command: ls -lh Description: Shows file sizes in a human-readable format (e.g., KB, MB). Example Output: drwxr-xr-x 2 user group 4.0K Aug 12 20:00 directory -rw-r--r-- 1 user group 1.2K Aug 12 20:00 file.txt Combine Options Command: ls -alh Description: Combines multiple options to list all files (including hidden ones) in long format with human-readable sizes. Example Output: drwxr-xr-x 5 user group 4.0K Aug 12 20:00. drwxr-xr-x 3 user group 4.0K Aug 12 20:00.. -rw-r--r-- 1 user group 1.2K Aug 12 20:00.bashrc drwxr-xr-x 2 user group 4.0K Aug 12 20:00 directory -rw-r--r-- 1 user group 1.2K Aug 12 20:00 file.txt Consider the following directory structure: ls -R /home/user Output: /home/user: dir1 file1.txt file4.txt /home/user/dir1: dir2 file2.txt /home/user/dir1/dir2: file3.txt The ls -x command in Linux lists directory contents in columns, sorted horizontally. This means that the files and directories are displayed in rows across the terminal window, rather than in a single column or sorted vertically. The ls -F command in Linux is used to append a character to each file name indicating the file type. This makes it easier to distinguish between different types of files and directories at a glance. Here’s what the appended characters mean: / for directories * for executable files @ for symbolic links | for FIFOs (named pipes) = for sockets #!/bin/bash # This script displays the current user, date and time, and the contents of the current directory echo "Current user: $(whoami)" echo "Current date and time: $(date)" echo "Contents of the current directory:" ls -l ls ls LS ls chap* ls –l ls –l chap* ls options ls -x output in multiple columns ls –F Identifying directories and executable ls -a list all files including hidden file starting with. ls –x dir _name Listing directory contents ls –xR Recursive listing  ls ls –d current directory ls –l list with long format -show permissions ls –la list long format including hidden file ls –s list file size ls –S sort by file size ls –t sort by time & date  who: Who are the users? The who command is used to display information about the users currently logged into the system. $ who user1 tty7 2024-08-12 09:00 user2 pts/0 2024-08-12 09:05 user1 and user2 are the usernames. tty7 and pts/0 are the terminals they are logged into. 2024-08-12 09:00 and 2024-08-12 09:05 are the login times. tty – Teletypewriter-tty devices represent physical or virtual terminals directly connected to the computer, such as a keyboard and monitor, or a serial connection. pts-Psuedo Terminal Slave-tty devices represent physical or virtual terminals directly connected to the computer, such as a keyboard and monitor, or a serial connection. who -H command in Unix/Linux is used to display the list of currently logged-in users, including a header line that describes each column of the output. This can be helpful for understanding the information presented. $who –H NAME LINE TIME COMMENT user1 tty7 2024-08-12 09:00 user2 pts/0 2024-08-12 09:05 NAME: The username of the logged-in user. LINE: The terminal line they are logged into. TIME: The time they logged in. COMMENT: Additional information, such as the host. who -u command, which provides more detailed information about each user, including idle time and the process ID of the user’s shell. $who –uH NAME LINE TIME IDLE PID COMMENT user1 tty7 2024-08-12 20:00. 1234 user2 pts/0 2024-08-12 20:10 00:05 5678 NAME: The username of the logged-in user. LINE: The terminal line. TIME: The time the user logged in. IDLE: The idle time (how long the user has been idle). PID: The process ID of the user’s shell. COMMENT: Additional information, such as the remote host.  date: Displaying the system date The date command is used to display and set the system date and time. Display Current Date and Time To display the current date and time, simply type: date This will output something like: Thu Aug 15 20:12:43 IST 2024 Format the Output You can format the output using various format specifiers. date +"Year: %Y, Month: %m, Day: %d" Year: 2024, Month: 08, Day: 15  The command can be used with suitable format specifiers $date +%m 08 $date +%h Aug $date +”%h %m” Aug 08 Other format specifiers are d- the day of the month (1 to 31) y-The last two digits of the year H, M, S – The Hour, Minute and Second D- date in format mm/dd/yy T- The time in the format hh:mm:ss date $date $date +%m $date +%h $date +”%H %M” $date +%d $date +%y $date +%S $date +%D $date +%T Display Past or Future Dates You can also display past or future dates using relative terms: date --date=“2 years ago” Thu Aug 15 23:10:04 IST 2024 date --date=“next Friday” Fri Aug 23 00:00:00 IST 2024  passwd: Changing your password If a users account is not set with the password or if it is already known to others then to change the password passwd command is used. $passwd passwd: Changing password for kumar Enter login password: ******** New password: ******** Re-enter new password:******** passwd(SYSTEM) : passwd successfully changed for kumar passwd $passwd To change the password.  cal: The Calendar cal command is used to see the calendar of any specific month or a complete year. cal [ [month] year ] Everything between the rectangular bracket is optional cal can be used without arguments in which it displays the current month $ cal $ cal 01 2024 Display the entire current year: cal -y This command shows the calendar for the entire current year. Display the previous, current, and next month: cal -3 This command shows the calendars for the previous, current, and next months. ps process status The ps command in UNIX and Linux is used to display information about the currently running processes. Without options: Displays processes associated with the current terminal. ps This will show columns like PID (Process ID), TTY (Terminal), TIME (CPU time), and CMD (Command). ps aux Displays all processes in a detailed format. This command is particularly useful for system monitoring and troubleshooting, as it provides a comprehensive view of all active processes.  cat command The cat command is a versatile tool used for various file-related operations Basic Usage Display File Contents: To display the contents of a file, use: cat filename.txt The contents of filename.txt will be displayed Concatenate Multiple Files: Combine and display the contents of multiple files: cat file1.txt file2.txt The output of file1.txt and file2.txt will be displayed Create a New File: Create a new file and add content to it: cat > newfile.txt This is a newfile created using cat command. ….. Ctrl+D to save the file Copy Contents to Another File: Copy the contents of one file to another: cat file1.txt > file2.txt Contents of file1.txt will be copied to the contents of file2.txt. The original content of file2.txt will be replaced by file1.txt contents Append Contents to a File: Append the contents of one file to another: cat file1.txt >> file2.txt Contents of file1.txt will be appended to the contents of file2.txt Display Line Numbers: Show line numbers along with the file content: cat -n filename.txt 1 line 1 contents 2 line 2 contents Suppress Repeated Empty Lines: Remove repeated empty lines in the output: cat -s filename.txt If filename.txt has repeated empty lines, those lines will be removed  wc command The wc command, short for “word count,” is a handy utility in Unix and Linux operating systems. It provides information about the number of lines, words, and characters in a file. Basic Syntax: To analyze a file, simply type: wc filename.txt cat > file3.txt This is a new file Output Columns: By default, Thanks wc displays four columns of information: wc file3.txt First column: Number of lines in the file. 2 6 26 file3.txt Second column: Number of words. Third column: Number of characters. Fourth column: File name. Options: -l: Prints the number of lines in the file. -w: Displays the word count. -c: Shows the byte count (number of bytes). -m: Counts characters. -L: Prints the length of the longest line in the file. cat > file3.txt This is a new file To count lines in a file: Thanks wc -l filename.txt ==🡺 2 file3.txt To count words: wc -w filename.txt ===🡺 6 file3.txt To count bytes: wc -c filename.txt ===🡺 26 file3.txt  Flexibility of command usage 1) Combining commands You can run multiple commands in a single line by separating them with a semicolon (;). This allows you to execute several commands sequentially. Here’s an example: command1; command2; command3 For instance, if you want to list the contents of a directory, create a new directory, and then navigate into it, you can do: ls; mkdir new_directory; cd new_directory Each command will run one after the other.  Flexibility of command usage Commands may be grouped together with parenthesis to redirect the output (date;ls –l testdir) > newlist Here newlist has the output of both the commands Shell understands that the command on each side of it needs to be processed separately ; is a metacharacter.  Flexibility of command usage 2) Command line can overflow or be split into multiple lines Command line can be entered even if the width may exceed the terminal width. Command overflows to the next line though it is a single logical line. Command line can be desirably split into multiple line. The shell issues a secondary prompt > to indicate command line is not complete. Ex: echo “This is >a multiline text”  Flexibility of command usage 3) Entering a command before previous command has finished UNIX lets us type a command at anytime , the system will interpret it. When a long program is run, the prompt is not shown till completed but still can accept commands and interpret. The input stored in buffer maintained by kernel. The command is passed to shell for interpretation after previous program has completed. Sun Aug 18 22:44:43 IST 2024 August 2024 $sleep 10 Su Mo Tu We Th Fr Sa 1 2 3 date; cal 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31  man : Browsing the manual pages online Man-Online Help facility command Man displays the documentation called man documentation of every command in the system. Ex: $man wc Displays first page Output sent to a pager program – displays one page at a time. In UNIX/Linux, the man command uses a pager to display the manual pages one screenful at a time. By default, the pager used is typically less, but it can also be configured to use other pagers like more. more: One of the oldest pagers, more allows you to scroll down through a text file one screen at a time. It has basic navigation features but cannot scroll backward. less: An advanced pager that allows both forward and backward navigation through text files. It includes additional features like search and is often the default pager on many systems For example, if you run: ls -l | more You can only move forward through the list of files. If you accidentally skip a screen, you cannot go back to it. In contrast, less allows both forward and backward navigation, even when used with pipes, making it more versatile for viewing long outputs. ls -l | less Navigation and Search Navigating and searching within a man page in Linux can be very efficient once you know the right commands. Navigation Spacebar: Move forward one page. b: Move backward one page. Enter: Move forward one line. Up/Down Arrow Keys: Move up or down one line. PgUp/PgDn: Move up or down one page. Home/End: Move to the beginning or end of the man page. Navigation and Search Searching /: Start a forward search. Type the search term and press Enter. Use n to go to the next occurrence and N to go to the previous occurrence. ?: Start a backward search. Type the search term and press Enter. Use n to go to the next occurrence and N to go to the previous occurrence. Exiting q: Quit the man page viewer. Understanding the man documentation Understanding a man page in Linux can be very helpful for getting detailed information about commands and their usage. A man page is divided into a number of compulsory and optional sections. Every command does not have all sections, but the first three(NAME, SYNOPSIS and DESCRIPTION) are generally seen in all man pages. Structure of a man Page NAME: The name of the command or function, followed by a brief description. SYNOPSIS: Shows the command syntax, including options and arguments. DESCRIPTION: Provides a detailed explanation of the command, its options, and its behavior. OPTIONS: Lists and explains the command-line options available for the command. EXAMPLES: (If available) Provides examples of how to use the command. SEE ALSO: References to related commands or documentation. AUTHOR: Information about the author(s) of the command or documentation. BUGS: Information about known bugs or issues. COPYRIGHT: Licensing information. The SYNOPSIS section of a man page provides a concise summary of how to use a command, including its options and arguments. Conventions in SYNOPSIS Brackets [ ]: Indicate optional elements. Example: [OPTION] means the option is not required. Ellipsis... : Indicates that the preceding element can be repeated. Example: FILE... means one or more files can be specified. Vertical Bar | : Represents a choice between options. Example: -a | -b means you can use either -a or -b Example: man ls Here’s a brief look at the man page for the ls command: NAME: ls - list directory contents SYNOPSIS: ls [OPTION]... [FILE]... DESCRIPTION: List information about the FILEs (the current directory by default). OPTIONS: -a, --all do not ignore entries starting with. -l use a long listing format Navigating a man Page Spacebar: Move forward one page. b: Move backward one page. Enter: Move forward one line. Up/Down Arrow Keys: Move up or down one line. /: Start a forward search. Type the search term and press Enter. n: Go to the next occurrence of the search term. N: Go to the previous occurrence of the search term. q: Quit the man page viewer. There is a separate section named EXIT STATUS which lists the possible error conditions and their numeric representation. These numbers can be of help when writing shell scripts in order to determine the actual cause of termination of a command Understanding Exit Status 0: The command was successful without any errors. Non-zero (1-255): The command failed. Different values can indicate different types of errors. Exit status: 0 if OK, 1 if minor problems (e.g., cannot access subdirectory), 2 if serious trouble (e.g., cannot access command-line argument). Checking Exit Status After running a command, you can check its exit status by typing: man ls echo $? Using man to understand man You can use man to understand itself by typing: man man This will show you the manual page for the man command, explaining how to use it and its various options. In UNIX/Linux, the man command uses a pager to display the manual pages one screenful at a time. By default, the pager used is typically less, but it can also be configured to use other pagers like more. You can change the default pager by setting the PAGER environment variable. For example, to use more instead of less, you can run: export PAGER=more The man –k, apropos and whatis command The man -k command in UNIX/Linux is used to search the manual page descriptions for keywords. It’s a helpful way to find commands related to a specific topic or function when you don’t know the exact command name. If you want to find commands related to “list,” you can use: man -k list This will return a list of commands and their short descriptions that include the keyword “list.” -k: This option stands for “keyword” and searches the short descriptions and names of manual pages for the specified keyword. Output: The output will include the command names and their brief descriptions, helping you identify which command might be useful for your needs. The man –k, apropos and whatis command apropos command helps to find out the files and commands associated with a given keyword. apropos list This will return all commands and their descriptions that include the keyword “list”. The man –k, apropos and whatis command The whatis command in Linux is used to display a brief, one-line description of a specified command from the system’s manual pages. This can be particularly useful when you need a quick summary of what a command does without reading through the entire manual page whatis ls This will return a short description of the ls command. Special Characters. (dot): Represents the current directory... (dot-dot): Represents the parent directory. / (forward slash): Root directory. ~ (tilde): Home directory of the current user. Knowing your terminal The who command will give us the information we are looking for. The one-letter command w requires less typing and provides more information. USER: The user name. TTY: The type of terminal they are logged in at FROM: The name of the remote host if this is a remote connection. LOGIN@: The time at which the user logged in. IDLE: Idle time.  JCPU: Joint CPU time, this is the CPU time used by all processes that have been attached to this tty. In other words, the total CPU time of this user in this logged in session. PCPU: Process CPU time, this is the CPU time used by the current process. The current process is named in the WHAT column. WHAT: The command line of this user’s current process. The tty command of terminal basically prints the file name of the terminal connected to standard input. tty is short of teletype, but popularly known as a terminal it allows you to interact with the system by passing on the data (you input) to the system, and displaying the output produced by the system Knowing your terminal To find out which terminal you’re currently using in a Linux environment, you can use the tty command. This command prints the file name of the terminal connected to the standard input. $ tty /dev/tty3 stty -Displaying its characteristics and setting characteristics The stty command in Linux is used to change and print terminal line settings. Display all settings: stty -a This prints all current settings in a human-readable form. Speed 38400 baud: Sets the terminal speed to 38400 baud. It means that the terminal communicates at a rate of 38400 bits per second. rows 26; columns 110: Configures the terminal to have 26 rows and 110 columns. intr = ^C: Sets the interrupt character to Ctrl+C. erase = ^?: Sets the erase character to Delete. kill = ^U: Sets the kill character to Ctrl+U (deletes the current line). eof = ^D: Sets the end-of-file character to Ctrl+D. Change the erase character: stty erase ^H This sets the erase character to backspace. Disable echo: stty -echo This turns off the echo, so characters typed are not displayed on the screen. Enable echo: stty echo This turns the echo back on. Displaying its characteristics and setting characteristics Backspace should erase characters: $ stty –echoe This command ensures that the backspace key erases characters. Enter password through a shell script: $ stty –echo This command disables the echo, so when you type your password, it won’t be displayed on the screen. Changing the interrupt key: $ stty intr \^c This sets the interrupt key to Ctrl+C. Changing the end of file: $ stty eof \^a This sets the end-of-file character to Ctrl+A. When everything else fails: $ stty sane This command resets the terminal to a sane state, undoing any changes that might have caused issues. Managing the non uniform behavior of terminals and keyboards Keystroke Function [Ctrl-h] Erase text [Ctrl-c] or [delete] Interrupts a command [Ctrl-d] Terminates login session or a program that expects its input from the key board [Ctrl-s] Stops scrolling of screen output and locks keyboard [Ctrl-q] Resumes scrolling of screen output and unlocks keyboard [Ctrl-u] Kills command line without executing it Managing the non uniform behavior of terminals and keyboards Keystroke Function [Ctrl-] Kills running command but creates a core file containing the memory image of the program [Ctrl-z] Suspends process and returns shell prompt [Ctrl-j] Alternative to [enter] [Ctrl-m] As above Stty sane Restores terminal to normal status bc command The bc command in Linux is a powerful tool used for performing mathematical calculations directly from the command line. It’s often referred to as a basic calculator and supports both integer and floating-point arithmetic. Basic Usage To start bc, simply type bc in your terminal: bc This opens an interactive prompt where you can type arithmetic operations. Common Options -l: Load the standard math library, enabling advanced mathematical functions(sine (s), cosine (c), exponential (e), natural logarithm (l) etc). -q: Quiet mode, which suppresses the welcome message. $bc 12+5 echo "12 + 5" | bc 17 17 11*3 33 echo "x=10; x*2" | bc [Ctrl-d] 20 $ $bc 12+5; 11*3 17 33 [Ctrl-d] $ In bc, the ibase and obase variables are used to set the input and output number bases, respectively. This allows you to convert numbers between different bases. Setting ibase and obase ibase: Sets the base for input numbers. obase: Sets the base for output numbers obase=2 8 1000 12 1100 script command The script command is used to record your terminal session, capturing all the input and output displayed on the terminal. This can be particularly useful for debugging, creating tutorials, or keeping a log of your activities. To start recording, simply type: script This will start a new session and save the output to a file named typescript by default. To stop recording, type: exit Here are some useful options for the script command: -a, --append: Append the output to the file instead of overwriting it. -c, --command: Run a specific command rather than an interactive shell. -q, --quiet: Run without showing start and stop messages. tput The tput command in Linux is used to control the terminal’s behavior and appearance like clearing the screen, moving the cursor, or changing text colors. Clear the Screen: tput clear This command clears everything on your terminal screen. Move the Cursor: tput cup 10 20 Moves the cursor to row 10, column 20. Change Text Color: tput setaf 1 Changes the text color to red (1 is the color code for red). Bold Text: tput bold Makes the text bold. Reset Text Formatting: tput sgr0 Resets any text formatting (like bold or color) back to normal. Here’s a simple script that uses tput to clear the screen, move the cursor, and print a message in bold: In this script: #!/bin/bash tput clear Clear the screen. tput cup 5 10 Move the cursor to row 5, column 10. tput bold Print “Hello, World!” in bold and red tput setaf 1 Reset the text formatting to normal. echo "Hello, World!" tput sgr0 # Reset text formatting uname The uname command in UNIX/Linux is used to display system information. Here are some common options you can use with uname: uname: Displays the kernel name. uname -a: Shows all available system information. uname -s: Displays the kernel name. uname -r: Shows the kernel release. uname -v: Displays the kernel version. uname -m: Shows the machine hardware name. uname -n: Displays the network node hostname. uname -p: Shows the processor type. uname -i: Displays the hardware platform. uname -o: Shows the operating system. root: The System Administrators Login The root account is the superuser or administrative account in UNIX/Linux systems. It has the highest level of access and permissions, allowing it to perform any action on the system, including modifying system files, installing software, and managing user accounts. Login: root Password :*******[Enter] #_ The prompt of root is # unlike $ or % used by non privileged users On modern systems most administrative commands are resident in /sbin and /usr/sbin and in Older systems most administrative commands are resident in /etc  root: The System Administrators Login roots path list is also different from one used by other users. Many standard scripts supplied with UNIX systems for system administrator work rely on the Bourne Shell for execution. Becoming thesuper user: su command. The su (substitute user) command in UNIX/Linux allows you to switch to another user’s account, typically the root user, within your current login session. This is useful for performing administrative tasks that require elevated privileges. Basic Usage To switch to the root user, simply type: su You will be prompted to enter the root password. Once authenticated, you will have root privileges. Becoming a super user On UNIX this special user is known as superuser or root(not to be confused with the root directory). Superuser can override file security and do almost anything he/she wants on the system (she cannot see your password, since it is encrypted, but she can change it). In fact, any user with a user id of 0 is a superuser  su: Acquiring Superuser Status A user can acquire superuser status with the su command if the user knows the root password User Juliet becomes superuser this way $ su Password : ******* # pwd /home/Juliet # prompt indicates that Juliet now has power of superuser Creating users environment Administrator can recreate the users environment without taking the login-password root – su –henry Su runs a separate sub-shell, so this mode is terminated by hitting [Ctrl-d] or using exit Becoming a super user Common Options Login Shell: To start a login shell with the environment similar to a real login, use: su - Specify Shell: To use a different shell, use the -s option: su -s /bin/bash Preserve Environment: To preserve the current user’s environment variables, use: su -p Run Command: To run a specific command as the root user without starting an interactive shell, use: su -c 'command' Example To switch to the root user and run the ls command: su -c 'ls' The administrator's privileges As a system administrator in UNIX/Linux, you have a wide range of powerful privileges. Here are some key ones: Change any user’s password: You can change any user’s password without knowing the existing one using the passwd command. For example: sudo passwd username Set the system clock: You can set the system clock using the date command. For example: sudo date MMDDhhmm[[CC]YY][.ss] Address all users concurrently: You can send a message to all logged-in users using the wall command. For example: sudo wall "System maintenance will start in 10 minutes." The administrator's privileges Limit the maximum size of files: You can limit the maximum size of files that users are permitted to create using the ulimit command. For example: ulimit -f size Control access to scheduling services: You can control users’ access to scheduling services like at and cron by editing the /etc/at.allow, /etc/at.deny, /etc/cron.allow, and /etc/cron.deny files. Control access to networking services: You can control users’ access to many networking services like FTP, SSH, etc., by configuring the respective service’s configuration files, such as /etc/ssh/sshd_config for SSH. The/etc/passwd and/etc/shadowfiles /etc/passwd is the key file in any Linux Unix system. The password file is a human-readable file that contains information about users on the system including their encrypted passwords Some systems don’t have encrypted passwords in this file if /etc/shadow file is generated. cat /etc/passwd /etc/passwd and /etc/shadow Username: Its user name being used by the user to login Password: Its password in an encrypted format.you see x instead of encrypted password since /etc/shadow file is generated on the system. The encrypted password is found in /etc/shadow file in such case. UID: Its user id. Its unique number assigned to every account on the system. GID: Its group id. Its unique number of groups of which account is member of. Comment: This field is introduced to have some descriptions against the account. This is purely for humans to identify/understand what related account is or to whom it belongs to Home directory: Its a directory where normally user lands into when he/she login. Login shell: This is a shell that will be spawn when the user successfully logs in. root:x:0:0:root:/root:/bin/bash root:$6$h./k6HFM$EmRq6TQl80Lq7BF5k0v9A8MvCZx5PddhczyUViwtfj.FWkl6xsE0Tt YJ 91 Commands to add, modify and delete users. To add a new user in Linux, you can use the adduser command sudo adduser john This will create a new user named john and prompt you to set a password and provide other details Commands to add, modify and delete users. The userdel command in Linux is used to delete a user account and its associated files. Here’s a quick guide on how to use it: Basic Syntax sudo userdel username Replace username with the actual username you want to delete.. It has only two options – -r Remove user’s home_dir & mail spool -f Removes user even if he/she logged in. Removes home_dir, mail spool & group of the same name even these are being shared by another user. Dangerous! Commands to add, modify and delete users. The usermod command in Linux is used to modify user accounts. Here are some common uses and options: Common Options Add a User to a Group: sudo usermod -a -G groupname username Use the -a (append) option to add the user to a new group without removing them from other groups. Change User’s Primary Group: sudo usermod -g groupname username Change User’s Home Directory: sudo usermod -d /new/home/dir -m username The -d option specifies the new home directory, and the -m option moves the content from the old home directory to the new one. Change User’s Login Shell: sudo usermod -s /path/to/new/shell username Lock a User Account: sudo usermod -L username This locks the user’s account by placing a ! in front of the encrypted password in the /etc/shadow file. Unlock a User Account: sudo usermod -U username Set Account Expiry Date: sudo usermod -e YYYY-MM-DD username End of Module-1