LINUX FUNDAMENTALS PDF

**LINUX FUNDAMENTALS** **THE OUTLINES** **Unit-1: An overview of the linux system** **Unit-2:** Understanding the filesystem **Unit-3:** The linux filesystem **Unit-4:** Working with file links in linux system **Unit-5:** Searching for files in linux system **Unit-6:** Working with users and...

**LINUX FUNDAMENTALS** **THE OUTLINES** **Unit-1: An overview of the linux system** **Unit-2:** Understanding the filesystem **Unit-3:** The linux filesystem **Unit-4:** Working with file links in linux system **Unit-5:** Searching for files in linux system **Unit-6:** Working with users and groups in linux system **Unit-7:** Working with file permissions in linux system **Unit-8:** Working with text files in linux system **Unit-9:** Working with vim text editor in linux system Linux is one of the major operating systems and is heavily used in organizations, all around the world. Learning how to use Linux is a core competency and will help you in your hacking journey, it teaches how not to just use Linux-based security tools, but how to use and exploit the operating system. This module will focus on getting you comfortable using Linux. Its fundamentals will help you learn all the essentials of the Linux command line required to get you started. Linux is not the name of a specific and full working OS, but implements only the essential inner core of an OS, which is referred to as the kernel. Most types of Linux OS do not cost anything, but provide thousands of programs and software to use completely free of charge. Most of these programs are also open source, which means you can view the exact blueprint of how the program has been created and it can be changed by anyone. One very important area in which Linux is very popular and is used heavily is managing many network services. These are programs that run in the background of your Linux server, continuously waiting for external events to respond with some kind of action or information. Examples are popular internet services such as a web server, which presents websites to the user; main servers for email communications; and also database servers to store and deliver any kind of data. As mentioned before, Linux is only the name of the kernel part, not a full working OS. To make it complete, you need to bundle it together with all kinds of programs, which is then referred to as a **distribution**. Nowadays, one can choose between an insane amount of different Linux distributions, all designed for a special purpose and all with their own pros and cons. The main difference between them is the software selected and bundled together with the Linux kernel. The most important Linux distribution families are Red Hat- and Debian-based Linux distributions. CentOS is one of the most important free Red Hat-based Linux server distributions out there at the moment. It\'s a very stable, secure, and reliable OS, which is why it\'s often used in running very critical networking services in enterprise environments. Also, it\'s good to know that the development of this OS is very strong and updates are well selected and suitably tested. The first chapter of this book is about installing Linux; here we will be starting easy by introducing you to the concepts of virtualization. Then, we will create a new CentOS 7 **virtual machine** (**VM**) using a free virtualization software called VirtualBox. **THE MEANING OF FILE AND ITS STRUCTURE TYPES** - All computer applications need to store and get back information. While a process is running, it can store a limited amount of information within its own address space. On the other hand, the storage capacity is restricted to the size of the virtual address space. For some applications this size is adequate, but for others, such as airline reservations, banking, or corporate record keeping, it is far too small. - A second problem with keeping information within a process address space is that when the process terminates, the information is lost. For many applications, (e.g., for databases), the information must be retained for weeks, months, or even forever. Having it vanish when the process using it terminates is unacceptable. Moreover, it must not go away when a computer crash kills the process. - A third problem is that it is often necessary for multiple processes to access (parts of) the information at the same time. If we have an online telephone directory stored inside the address space of a single process, only that process can access it. The way to solve this problem is to make the information itself independent of any one process. Therefore we have three essential requirements for long-term information storage: \- Files are logical units of information created by processes. A disk will generally contains thousands or even millions of them, each one independent of the others. In reality, if you think of each file as a kind of address space, you are not that far off, except that they are used to model the disk instead of modeling the RAM. **NAMING OF FILES** - Files are an abstraction mechanism. They provide a way to store information on the disk and read it back later. This must be done in such a way as to shield the user from the details of how and where the information is stored, and how the disks really work. Some typical file extensions **FILE STRUCTURE** ------------------ - Files can be structured in any of various ways. Three common possibilities are s in Figure  - Having the operating system regard files as nothing more than byte sequences provides the maximum flexibility. User programs can put anything they want in their files and name them any way that is convenient. The operating system does not help, but it also does not get in the way. For users who want to do unusual things, the latter can be very important. All versions of UNIX, MS-DOS, and Windows use this file model. - The third kind of file structure is shown in Figure 2(c). In this organization, a file consists of a tree of records, not necessarily all the same length, each containing a key field in a fixed position in the record. The tree is sorted on the key field, to allow rapid searching for a particular key. TYPES OF FILE ATTRIBUTES AND FILE EXTENSIONS ============================================ - Every file has a name and its data. Moreover, all operating systems associate other information with each file, for instance, the date and time the file was last modified and the file\'s size. We will call these extra items the file\'s attributes. Some people call them metadata. The list of attributes varies significantly from system to system. The table of Figure 1 shows some of the possibilities, but other ones also exist. No existing system has all of these, but each one is present in some system. - The flags are bits or short fields that control or enable some particular property. Hidden files, for instance, do not appear in listings of all the files. The archive flag is a bit that keeps track of whether the file has been backed up recently. The backup program clears it, and the operating system sets it whenever a file is changed. Thus, the backup program can tell which files need backing up. The temporary flag allows a file to be marked for automatic deletion when the process that created it terminates. - The record length, key position, and key length fields are only present in files whose records can be looked up using a key. They provide the information required to find the keys. - The various times keep track of when the file was created, most recently accessed, and most recently modified. These are useful for a variety of purposes. For instance, a source file that has been modified after the creation of the corresponding object file needs to be recompiled. These fields provide the necessary information. - The current size tells how big the file is at present. Some old mainframe operating systems require the maximum size to be specified when the file is created, in order to let the operating system reserve the maximum amount of storage in advance. Workstation and personal computer operating systems are clever enough to do without this feature. FILE OPERATIONS =============== **Create** With this command, the file is created with no data. The purpose of the call is to announce that the file is corning and to set some of the attributes. **Delete** When the file is no longer required, it has to be deleted to free up disk space. This is always the system call for this purpose. **Open** Before using a file, a process must open it. The purpose of the open call is to allow the system to fetch the attributes and list of disk addresses into main memory for rapid access on later calls. **Close** When all the accesses are finished, the attributes and disk addresses are no longer required, so the file should be closed to free up internal table space. Many systems encourage this by imposing a maximum number of open files on processes. A disk is written in blocks, and closing a file forces writing of the file\'s last block, even though that block may not be entirely full yet. **Read** Data are read from file. Generally, the bytes come from the current position. The caller must specify how many data are required and must also provide a buffer to put them in. **Write** Data are written to the file again, generally at the current position. If the current position is the end of the file, the file\'s size increases. If the current position is in the middle of the file, existing data are overwritten and lost forever. **Append** This call is a restricted form of write. It can only add data to the end of the file. Systems that provide a minimal set of system calls do not generally have append, but many systems provide several ways of doing the same thing, and these systems sometimes have append. **Seek** For random access files, a method is required to specify from where to take the data. One common approach is a system call, seek, that repositions the file pointer to a particular place in the file. After this call has completed, data can be read from, or written to, that position. **Get Attributes** Processes often need to read file attributes to do their work. For instance, the UNIX make program is commonly used to manage software development projects consisting of many source files. When make is called, it examines the modification times of all the source and object files and arranges for the minimum number of compilations required to bring everything up to date. To do its job, it must look at the attributes, namely, the modification times. **Set Attributes** Some of the attributes are user settable and can be changed after the file has been created. This system call makes that possible. The protection mode information is an obvious example. Most of the flags also fall in this category. **Rename** It frequently happens that a user needs to change the name of an existing file. This system call makes that possible. It is not always strictly necessary, because the file can generally be copied to a new file with the new name, and the old file then deleted. **FILE TYPES** File type refers to the ability of the operating system to distinguish different types of file such as text files source files and binary files etc. Many operating systems support many types of files. Operating system like MS-DOS and UNIX have the following types of files − 1. Ordinary Files -------------- - These are the files that contain user information. - These may have text, databases or executable program. - The user can apply various operations on such files like add, modify, delete or even remove the entire file. 2. Directory Files --------------- - These files contain list of file names and other information related to these files. 3. Special Files ------------- - These files are also known as device files. - These files represent physical device like disks, terminals, printers, networks, tape drive etc. NB: These files are of two types: --------------------------------- - **Character special files** − data is handled character by character as in case of terminals or printers. - **Block special files** − data is handled in blocks as in the case of disks and tapes. FILE ACCESS METHODS =================== **NB: That is simply known as File Access Mechanisms** File access mechanism refers to the manner in which the records of a file may be accessed. There are several ways to access files and they are: - Sequential access - Direct/Random access - Indexed sequential access Sequential Access ----------------- Direct/Random Access -------------------- - Random access file organization provides, accessing the records directly. - Each record has its own address on the file with by the help of which it can be directly accessed for reading or writing. - The records need not be in any sequence within the file and they need not be in adjacent locations on the storage medium. Indexed sequential access ------------------------- - This mechanism is built up on base of sequential access. - An index is created for each file which contains pointers to various blocks. - Index is searched sequentially and its pointer is used to access the file directly. - Contiguous Allocation - Linked Allocation - Indexed Allocation Contiguous Allocation --------------------- - Each file occupies a contiguous address space on disk. - Assigned disk address is in linear order. - Easy to implement. - External fragmentation is a major issue with this type of allocation technique. Linked Allocation ----------------- - Each file carries a list of links to disk blocks. - Directory contains link / pointer to first block of a file. - No external fragmentation - Effectively used in sequential access file. - Inefficient in case of direct access file. Indexed Allocation ------------------ - Provides solutions to problems of contiguous and linked allocation. - An index block is created having all pointers to files. - Each file has its own index block which stores the addresses of disk space occupied by the file. - Directory contains the addresses of index blocks of files. FILE DIRECTORY ============== Single-Level Directory Systems ------------------------------ The simplest form of directory system is having one directory containing all the files. Sometimes it is called the root directory, but since it is the only one, the name does not matter much. An example of a system with one directory is given in Figure 1. Here the directory includes four files. The advantages of this scheme are its simplicity and the ability to locate files quickly - there is only one place to look, after all. It is normally used on simple embedded devices such as telephones, digital cameras, and some portable music players. **TYPES OF FILE SYSTEM** a. CD ROM file system b. MS DOS file system c. UNIX V7 file system **CD-ROM File Systems** ----------------------- - As our first instance of a file system, let us examine the file systems used on CD-ROMs. These systems are particularly simple because they were designed for write-once media. Among other things, for instance, they have no provision for keeping track of free blocks because on a CD-ROM files cannot be freed or added after the disk has been manufactured. Below we will take a look at the main CD-ROM file system type and two extensions to it. Some years after the CD-ROM made its debut, the CD-R (CD Recordable) was introduced. Unlike the CD-ROM, it is possible to add files after the initial burning, but these are simply appended to the end of the CD-R. Files are never removed (although the directory can be updated to hide existing files). As a consequence of this \"append-only\" file system, the essential properties are not changed. Particularly, all the free space is in one contiguous chunk at the end of the CD. **The ISO 9660 File System** ---------------------------- - The most common standard for CD-ROM file systems was adopted as an International Standard in 1988 under the name ISO 9660. Virtually every CD-ROM currently on the market is compatible with this standard, sometimes with the extensions to be discussed below. One of the goals of this standard was to make every CD-ROM readable on every computer, independent of the byte ordering used and independent of the operating system used. As a result, some limitations were placed on the file system to make it possible for the weakest operating systems then in use (such as MS-DOS) to read it. CD- ROMs do not have concentric cylinders the way magnetic disks do. Instead there is a single continuous spiral containing the bits in a linear sequence (although seeks across the spiral are possible). The bits along the spiral are divided into logical blocks (also called logical sectors) of 2352 bytes. Some of these are for preambles, error correction, and other overhead. The payload portion of each logical block is 2048 bytes. When used for music, CDs have leadins, leadouts, and intertrack gaps, but these are not used for data CD-ROMs. - **Rock Ridge Extensions** ------------------------- - As we have seen, ISO 9660 is highly restrictive in several ways. Shortly after it came out, people in the UNIX community began working on an extension to make it possible to represent UNIX file systems on a CD-ROM. These extensions were named Rock Ridge, after a town in the Gene Wilder movie Blazing Saddles, probably because one of the committee members liked the film. The extensions use the System use field in order to make Rock Ridge CD-ROMs readable on any computer. All the other fields retain their normal ISO 9660 meaning. Any system not aware of the Rock Ridge extensions just ignores them and sees a normal CD-ROM. - PX - POSIX attributes. - PN - Major and minor device numbers. - SL - Symbolic link. - NM - Alternative name. - CL - Child location. - PL - Parent location. - RE - Relocation. - TF - Time stamps. **Joliet Extensions** --------------------- - The UNIX community was not the only group that wanted a way to extend ISO 9660. Microsoft also found it too restrictive (although it was Microsoft\'s own MS-DOS that caused most of the restrictions in the first place). Therefore Microsoft invented some extensions that were called Joliet. They were designed to allow Windows file systems to be copied to CD-ROM and then restored, in precisely the same way that Rock Ridge was designed for UNIX. Virtually all programs that run under Windows and use CD-ROMs support Joliet, including programs that burn CD-recordable. Generally, these programs offer a choice between the various ISO 9660 levels and Joliet. 1. Long file names. 2. Unicode character set. 3. Directory names with extensions Directory nesting deeper than eight levels. **MS-DOS File System** ---------------------- - The MS-DOS file system is the one the first IBM PCs came with. It was the main file system up through Windows 98 and Windows ME. It is still supported on Windows 2000, Windows XP, and Windows Vista, although it is no longer standard on new PCs now except for floppy disks. However, it and an extension of it (FAT-32) have become widely used for many embedded systems. Most digital cameras use it. Many MP3 players use it exclusively. The popular Apple iPod uses it as the default file system, although knowledgeable hackers can reformat the iPod and install a different file system. - In this way the number of electronic devices using the MS-DOS file system is vastly larger now than at any time in the past, and certainly much larger than the number using the more modern NTFS file system. For that reason alone, it is worth looking at in some detail. To read a file, an MS-DOS program must first make an open system call to get a handle for it. The open system call specifies a path, which may be either absolute or relative to the current working directory. The path is looked up component by component until the final directory is located and read into memory. It is then searched for the file to be opened. **UNIX V7 File System** ----------------------- - The early versions of UNIX had a fairly sophisticated multiuser file system since it was derived from MULTICS. Below we will talk about the V7 file system, the one for the PDP- 11 that made UNIX famous. The file system is in the form of a tree starting at the root directory, with the addition of links, forming a directed acyclic graph. File names are up to 14 characters and can contain any ASCII characters except / (because that is the separator between components in a path) and NUL (because that is used to pad out names shorter than 14 characters). NUL has the numerical value of 0. A UNIX directory entry includes one entry for each file in that directory. Each entry is very simple because UNIX uses the **i-node** scheme. A directory entry includes only two fields: the file name (14 bytes) and the number of the **i-node** for that file (2 bytes). These parameters limit the number of files per fil **UNIT 3** **LINUX DISTRIBUTIONS (DISTROS)** **Introduction** Other operating systems like Microsoft combine each bit of codes internally and release it as a single package. You have to choose from one of the version they offer. But Linux is different from them. Different parts of Linux are developed by different organizations. Different parts include kernel, shell utilities, X server, system environment, graphical programs, etc. If you want you can access the codes of all these parts and assemble them yourself. But its not an easy task seeking a lot of time and all the parts has to be assembled correctly in order to work properly. From here on distribution (also called as distros) comes into the picture. They assemble all these parts for us and give us a compiled operating system of Linux to install and use. - A Linux distribution is an OS made through a software collection that contains the Linux kernel and a package management system often. - Usually, Linux users obtain their OS by downloading a Linux distribution, available for a range of systems from embedded devices (e.g., ***OpenWrt***) to robust supercomputers (e.g., Rocks Cluster Distribution). - A Linux distribution is composed of a Linux kernel, GNU libraries and tools, other software, a window system, documentation, a desktop environment, and a window manager. - Almost every added software is open-source and free and becomes available both as in source code and compiled binary form, permitting changes to the actual software. - Optionally, Linux distributions add a few proprietary software that might not be available in the source code form, like binary blocks needed for a few device drivers. **History of Linux Distributions** Linus Torvalds integrated the Linux kernel and shared its first version, 0.01, in 1991. Initially, Linux was distributed as only source code, and after that, as a combination of downloadable floppy disk images. Distributions started simplifying the installation procedure as it was complicated, specifically during the growing numbers of available software. Users admired Linux distributions as replacements to the Microsoft Windows and DOS OSes on proprietary Unix versions, Apple Macintosh Mac OS, and IBM PC compatible systems. Almost every early adopter was familiar with Unix from school or work. They accepted Linux distributions for low cost and the source code availability for all or most of their software. Linux has become more famous in embedded and server device markets as compared to the desktop market as of 2017. It is utilized on over 50% of web servers. **Trends and Types** **Linux distributions might be:** - Non-commercial or commercial - Developed for home users, power users, or enterprise users - Supported on two or more types of platform or hardware-specific, even to the certification extension via platform vendor - Developed for embedded, desktop, or server devices - Highly specialized or general purpose toward particular machine functionalities (e.g., computer clusters, network routers, and firewalls) - Targeted at particular user groups, e.g., by language internationalization and localization or by including several scientific computing and music production packages - Primarily, built for comprehensiveness, portability, usability, or security\ Rolling release or standard release The Linux distribution diversity is because of the technical, philosophical, and organizational variation among users and vendors. Free software licensing defines that users having sufficient interest and knowledge can customize the existing distributions or create one to match their needs. **Linux Distributions List** There are on an average six hundred Linux distributors providing different features. Here, we\'ll discuss about some of the popular Linux distros today. **1) Ubuntu** It came into existence in 2004 by Canonical and quickly became popular. Canonical wants Ubuntu to be used as easy graphical Linux desktop without the use of command line. It is the most well-known Linux distribution. Ubuntu is a next version of Debian and easy to use for newbies. It comes with a lots of pre-installed apps and easy to use repositories libraries. Earlier, Ubuntu uses GNOME2 desktop environment but now it has developed its own unity desktop environment. It releases every six months and currently working to expand to run on tablets and smartphones. **2) Linux Mint** Mint is based on Ubuntu and uses its repository software so some packages are common in both. Earlier it was an alternative of Ubuntu because media codecs and proprietary software are included in mint but was absent in Ubuntu. But now it has its own popularity and it uses cinnamon and mate desktop instead of Ubuntu\'s unity desktop environment. **3) Debian** Debian has its existence since 1993 and releases its versions much slowly then Ubuntu and mint. This makes it one of the most stable Linux distributor. Ubuntu is based on Debian and was founded to improve the core bits of Debian more quickly and make it more user friendly. Every release name of Debian is based on the name of the movie Toy Story. **4) Red Hat Enterprise / CentOS** Red hat is a commercial Linux distributor. These products are red hat enterprise Linux (RHEL) and Fedora which are freely available. RHEL is well tested before release and supported till seven years after the release, whereas, fedora provides faster update and without any support. Red hat uses trademark law to prevent their software from being redistributed. CentOS is a community project that uses red hat enterprise Linux code but removes all its trademark and make it freely available. In other words, it is a free version of RHEL and provide a stable platform for a long time. **5) Fedora** It is a project that mainly focuses on free software and provides latest version of software. It doesn\'t make its own desktop environment but used \'upstream\' software. By default it has GNOME3 desktop environment. It is less stable but provides the latest stuff. **Choosing a Linux Distro** **Distribution** **Why To Use** -------------------- ------------------------------------------------------------------------------------ UBuntu It works like Mac OS and easy to use. Linux mint It works like windows and should be use by new comers. Debian It provides stability but not recommended to a new user. Fedora If you want to use red hat and latest software. Red hat enterprise To be used commercially. CentOS If you want to use red hat but without its trademark. OpenSUSE It works same as Fedora but slightly older and more stable. Arch Linux It is not for the beginners because every package has to be installed by yourself. **UNIT 4** This article is intended to be a very high-level discussion of Linux filesystem concepts. It is not intended to be a low-level description of how a particular filesystem type, such as EXT4, works, nor is it intended to be a tutorial of filesystem commands. Every general-purpose computer needs to store data of various types on a hard disk drive (HDD) or some equivalent, such as a USB memory stick. There are a couple reasons for this. First, RAM loses its contents when the computer is switched off. There are non-volatile types of RAM that can maintain the data stored there after power is removed (such as flash RAM that is used in USB memory sticks and solid state drives), but flash RAM is much more expensive than standard, volatile RAM like DDR3 and other, similar types. The second reason that data needs to be stored on hard drives is that even standard RAM is still more expensive than disk space. Both RAM and disk costs have been dropping rapidly, but RAM still leads the way in terms of cost per byte. A quick calculation of the cost per byte, based on costs for 16GB of RAM vs. a 2TB hard drive, shows that the RAM is about 71 times more expensive per unit than the hard drive. A typical cost for RAM is around \$0.0000000043743750 per byte today. For a quick historical note to put present RAM costs in perspective, in the very early days of computing, one type of memory was based on dots on a CRT screen. This was very expensive at about \$1.00 *per bit*! **DEFINITIONS** You may hear people talk about filesystems in a number of different and confusing ways. The word itself can have multiple meanings, and you may have to discern the correct meaning from the context of a discussion or document. I will attempt to define the various meanings of the word \"filesystem\" based on how I have observed it being used in different circumstances. Note that while attempting to conform to standard \"official\" meanings, my intent is to define the term based on its various usages. These meanings will be explored in greater detail in the following sections of this article. 1. The entire Linux directory structure starting at the top (/) root directory. 2. A specific type of data storage format, such as EXT3, EXT4, BTRFS, XFS, and so on. Linux supports almost 100 types of filesystems, including some very old ones as well as some of the newest. Each of these filesystem types uses its own metadata structures to define how the data is stored and accessed. 3. A partition or logical volume formatted with a specific type of filesystem that can be mounted on a specified mount point on a Linux filesystem. **BASIC FILESYSTEM FUNCTIONS** Disk storage is a necessity that brings with it some interesting and inescapable details. Obviously, a filesystem is designed to provide space for non-volatile storage of data; that is its ultimate function. However, there are many other important functions that flow from that requirement. All filesystems need to provide a namespace---that is, a naming and organizational methodology. This defines how a file can be named, specifically the length of a filename and the subset of characters that can be used for filenames out of the total set of characters available. It also defines the logical structure of the data on a disk, such as the use of directories for organizing files instead of just lumping them all together in a single, huge conglomeration of files. Once the namespace has been defined, a metadata structure is necessary to provide the logical foundation for that namespace. This includes the data structures required to support a hierarchical directory structure; structures to determine which blocks of space on the disk are used and which are available; structures that allow for maintaining the names of the files and directories; information about the files such as their size and times they were created, modified or last accessed; and the location or locations of the data belonging to the file on the disk. Other metadata is used to store high-level information about the subdivisions of the disk, such as logical volumes and partitions. This higher-level metadata and the structures it represents contain the information describing the filesystem stored on the drive or partition, but is separate from and independent of the filesystem metadata. Filesystems also require an Application Programming Interface (API) that provides access to system function calls which manipulate filesystem objects like files and directories. APIs provide for tasks such as creating, moving, and deleting files. It also provides algorithms that determine things like where a file is placed on a filesystem. Such algorithms may account for objectives such as speed or minimizing disk fragmentation. Modern filesystems also provide a security model, which is a scheme for defining access rights to files and directories. The Linux filesystem security model helps to ensure that users only have access to their own files and not those of others or the operating system itself. The final building block is the software required to implement all of these functions. Linux uses a two-part software implementation as a way to improve both system and programmer efficiency.  **Figure 1: The Linux two-part filesystem software implementation.** The first part of this two-part implementation is the Linux virtual filesystem. This virtual filesystem provides a single set of commands for the kernel, and developers, to access all types of filesystems. The virtual filesystem software calls the specific device driver required to interface to the various types of filesystems. The filesystem-specific device drivers are the second part of the implementation. The device driver interprets the standard set of filesystem commands to ones specific to the type of filesystem on the partition or logical volume. **DIRECTORY STRUCTURE** ----------------------- **Directory** **Description** --------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- / (root filesystem) The root filesystem is the top-level directory of the filesystem. It must contain all of the files required to boot the Linux system before other filesystems are mounted. It must include all of the required executables and libraries required to boot the remaining filesystems. After the system is booted, all other filesystems are mounted on standard, well-defined mount points as subdirectories of the root filesystem. /bin The /bin directory contains user executable files. /boot Contains the static bootloader and kernel executable and configuration files required to boot a Linux computer. /dev This directory contains the device files for every hardware device attached to the system. These are not device drivers, rather they are files that represent each device on the computer and facilitate access to those devices. /etc Contains the local system configuration files for the host computer. /home Home directory storage for user files. Each user has a subdirectory in /home. /lib Contains shared library files that are required to boot the system. /media A place to mount external removable media devices such as USB thumb drives that may be connected to the host. /mnt A temporary mountpoint for regular filesystems (as in not removable media) that can be used while the administrator is repairing or working on a filesystem. /opt Optional files such as vendor supplied application programs should be located here. /root This is not the root (/) filesystem. It is the home directory for the root user. /sbin System binary files. These are executables used for system administration. /tmp Temporary directory. Used by the operating system and many programs to store temporary files. Users may also store files here temporarily. Note that files stored here may be deleted at any time without prior notice. /usr These are shareable, read-only files, including executable binaries and libraries, man files, and other types of documentation. /var Variable data files are stored here. This can include things like log files, MySQL, and other database files, web server data files, email inboxes, and much more. As a usually very organized Virgo, I like things stored in smaller, organized groups rather than in one big bucket. The use of directories helps me to be able to store and then locate the files I want when I am looking for them. Directories are also known as folders because they can be thought of as folders in which files are kept in a sort of physical desktop analogy. In Linux and many other operating systems, directories can be structured in a tree-like hierarchy. The Linux directory structure is well defined and documented in the [Linux Filesystem Hierarchy Standard](http://www.pathname.com/fhs/) (FHS). Referencing those directories when accessing them is accomplished by using the sequentially deeper directory names connected by forward slashes (/) such as /var/log and /var/spool/mail. These are called paths. The following table provides a very brief list of the standard, well-known, and defined top-level Linux directories and their purposes. **Table 1: The top level of the Linux filesystem hierarchy** The directories and their subdirectories shown in Table 1, along with their subdirectories that have a teal background are considered an integral part of the root filesystem. That is, they cannot be created as a separate filesystem and mounted at startup time. This is because they (specifically, their contents) must be present at boot time in order for the system to boot properly. The /media and /mnt directories are part of the root filesystem, but they should never contain any data. Rather, they are simply temporary mount points. The remaining directories, those that have no background color in Table 1 do not need to be present during the boot sequence, but will be mounted later, during the startup sequence that prepares the host to perform useful work. Be sure to refer to the official [Linux Filesystem Hierarchy Standard](http://www.pathname.com/fhs/) (FHS) web page for details about each of these directories and their many subdirectories. Wikipedia also has a good description of the [FHS](https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard). This standard should be followed as closely as possible to ensure operational and functional consistency. Regardless of the filesystem types used on a host, this hierarchical directory structure is the same. **LINUX UNIFIED DIRECTORY STRUCTURE** ------------------------------------- In some non-Linux PC operating systems, if there are multiple physical hard drives or multiple partitions, each disk or partition is assigned a drive letter. It is necessary to know on which hard drive a file or program is located, such as C: or D: Then you issue the drive letter as a command, **D:** for example, to change to the D: drive, and then you use the **cd** command to change to the correct directory to locate the desired file. Each hard drive has its own separate and complete directory tree. The Linux filesystem unifies all physical hard drives and partitions into a single directory structure. It all starts at the top--the root (/) directory. All other directories and their subdirectories are located under the single Linux root directory. This means that there is only one single directory tree in which to search for files and programs. This can work only because a filesystem, such as /home, /tmp, /var, /opt, or /usr can be created on separate physical hard drives, a different partition, or a different logical volume from the / (root) filesystem and then be mounted on a mountpoint (directory) as part of the root filesystem tree. Even removable drives such as a USB thumb drive or an external USB or ESATA hard drive will be mounted onto the root filesystem and become an integral part of that directory tree. One good reason to do this is apparent during an upgrade from one version of a Linux distribution to another, or changing from one distribution to another. In general, and aside from any upgrade utilities like dnf-upgrade in Fedora, it is wise to occasionally reformat the hard drive(s) containing the operating system during an upgrade to positively remove any cruft that has accumulated over time. If /home is part of the root filesystem it will be reformatted as well and would then have to be restored from a backup. By having /home as a separate filesystem, it will be known to the installation program as a separate filesystem and formatting of it can be skipped. This can also apply to /var where database, email inboxes, website, and other variable user and system data are stored. There are other reasons for maintaining certain parts of the Linux directory tree as separate filesystems. For example, a long time ago, when I was not yet aware of the potential issues surrounding having all of the required Linux directories as part of the / (root) filesystem, I managed to fill up my home directory with a large number of very big files. Since neither the /home directory nor the /tmp directory were separate filesystems but simply subdirectories of the root filesystem, the entire root filesystem filled up. There was no room left for the operating system to create temporary files or to expand existing data files. At first, the application programs started complaining that there was no room to save files, and then the OS itself started to act very strangely. Booting to single-user mode and clearing out the offending files in my home directory allowed me to get going again. I then reinstalled Linux using a pretty standard multi-filesystem setup and was able to prevent complete system crashes from occurring again. I once had a situation where a Linux host continued to run, but prevented the user from logging in using the GUI desktop. I was able to log in using the command line interface (CLI) locally using one of the [virtual consoles](https://en.wikipedia.org/wiki/Virtual_console), and remotely using SSH. The problem was that the /tmp filesystem had filled up and some temporary files required by the GUI desktop could not be created at login time. Because the CLI login did not require files to be created in /tmp, the lack of space there did not prevent me from logging in using the CLI. In this case, the /tmp directory was a separate filesystem and there was plenty of space available in the volume group the /tmp logical volume was a part of. I simply [expanded the /tmp logical volume](https://opensource.com/business/16/9/linux-users-guide-lvm) to a size that accommodated my fresh understanding of the amount of temporary file space needed on that host and the problem was solved. Note that this solution did not require a reboot, and as soon as the /tmp filesystem was enlarged the user was able to login to the desktop. Another situation occurred while I was working as a lab administrator at one large technology company. One of our developers had installed an application in the wrong location (/var). The application was crashing because the /var filesystem was full and the log files, which are stored in /var/log on that filesystem, could not be appended with new messages due to the lack of space. However, the system remained up and running because the critical / (root) and /tmp filesystems did not fill up. Removing the offending application and reinstalling it in the /opt filesystem resolved that problem. **FILESYSTEM TYPES** -------------------- Linux supports reading around 100 partition types; it can create and write to only a few of these. But it is possible---and very common---to mount filesystems of different types on the same root filesystem. In this context we are talking about filesystems in terms of the structures and metadata required to store and manage the user data on a partition of a hard drive or a logical volume. The complete list of filesystem partition types recognized by the Linux **fdisk** command is provided here, so that you can get a feel for the high degree of compatibility that Linux has with very many types of systems. 0 Empty 24 NEC DOS 81 Minix / old Lin bf Solaris 1 FAT12 27 Hidden NTFS Win 82 Linux swap / So c1 DRDOS/sec (FAT- 2 XENIX root 39 Plan 9 83 Linux c4 DRDOS/sec (FAT- 3 XENIX usr 3c PartitionMagic 84 OS/2 hidden or c6 DRDOS/sec (FAT- 4 FAT16 \ 8. Hfsplus 9. Minix 10. M**sdos 11.** Ntfs 12. Reiserfs 13. **Yfat 13.** Xfs Other distributions support creating different filesystem types. For example, CentOS 6 supports creating only those filesystems highlighted in bold in the above list. **MOUNTING** ------------ The term \"to mount\" a filesystem in Linux refers back to the early days of computing when a tape or removable disk pack would need to be physically mounted on an appropriate drive device. After being physically placed on the drive, the filesystem on the disk pack would be logically mounted by the operating system to make the contents available for access by the OS, application programs and users. A mount point is simply a directory, like any other, that is created as part of the root filesystem. So, for example, the home filesystem is mounted on the directory /home. Filesystems can be mounted at mount points on other non-root filesystems but this is less common. The Linux root filesystem is mounted on the root directory (/) very early in the boot sequence. Other filesystems are mounted later, by the Linux startup programs, either **rc** under SystemV or by **system** in newer Linux releases. Mounting of filesystems during the startup process is managed by the /etc/fstab configuration file. An easy way to remember that is that fstab stands for \"file system table,\" and it is a list of filesystems that are to be mounted, their designated mount points, and any options that might be needed for specific filesystems. Filesystems are mounted on an existing directory/mount point using the **mount** command. In general, any directory that is used as a mount point should be empty and not have any other files contained in it. Linux will not prevent users from mounting one filesystem over one that is already there or on a directory that contains files. If you mount a filesystem on an existing directory or filesystem, the original contents will be hidden and only the content of the newly mounted filesystem will be visible. **UNIT 5** **WORKING WITH FILE LINKS IN LINUX** Linux is, without a doubt, one of the single most flexible operating system platforms on the planet. With the flagship open source ecosystem, there is almost nothing you cannot do. What makes Linux so flexible? The answer to that question will depend on your needs. Suffice it to say, the list of answers is significant and starts from the kernel and works its way out to the desktop environment. This flexibility was built into the operating system from the beginning, borrowing quite a lot of features from UNIX. One such feature is links. **WHAT DOES LINKS MEAN** Links are a very handy way to create a shortcut to an original directory. Links are used in many instances: Sometimes to create a convenient path to a directory buried deep within the file hierarchy; other uses for links include: - Linking libraries - Making sure files are in constant locations (without having to move the original) - Keeping a "copy" of a single file in multiple locations ### **ARE LINKS "SHORTCUTS"?** In a way, yes...but not exactly. Within the realm of Linux, there's more to links than just creating a shortcut to another location. Consider this: A shortcut is simply a pseudo-file that points to the original location of the file. For instance, create a shortcut on the Windows desktop to a particular folder and, when you click that icon, it will automatically open your file manager in the original location. On Linux, when you create a link in Linux, you click on that link and it will open the link in the exact location in which it was created. Let me explain. Say, for instance, you have an external drive, attached to your Windows machine. On that drive is a folder called Music. If you create a shortcut to the directory on your desktop, when you click to open the shortcut, your file manager will open to the Music directory on your external drive. Now, say you have that drive attached to a Linux machine. That drive is mounted to, say, */data* and on that drive is the folder *Music*. You create a link to that location in your home directory---so you how have a link from *\~/Music* that points to */data/Music*. If you open the shortcut in your home directory, it opens the file manager in *\~/Music*, instead of */data/Music*. Any changes you make in *\~/Music* will automatically be reflected in /data/Music. And that is the big difference. ### ### **TYPES OF LINKS** In Linux there are two different types of links: - Hard links - Symbolic links The difference between the two are significant. With hard links, you can only link to files (and not directories); you cannot reference a file on a different disk or volume, and they reference the same inode as the original source. A hard link will continue to remain usable, even if the original file is removed. Symbolic links, on the other hand, can link to directories, reference a file/folder on a different disk or volume, will exist as a broken (unusable) link if the original location is deleted, reference abstract filenames and directories (as opposed to physical locations), and are given their own, unique inode. Now comes the fun part. How do you work with links? Let's find out how to create both hard and symbolic links. ### **Working with Hard Links** We're going to make this very simple. The basic command structure for creating a hard link is: *ln SOURCE LINK* Where SOURCE is the original file and LINK is the new file you will create that will point to the original source. So let's say we want to create a link pointing to */data/file1* and we want to create the link in the *\~data/file1*. If you open up both files, you will see they have the exact same contents. If you alter the contents in one, the changes will reflect in both. One of the benefits of using hard links is that if you were to delete */data/file1*, *\~/file1* would still remain. If you want to simply remove the link, you can use the *rm* command like so: *rm \~/file1* ### **Working with Symbolic Links** The command structure for symbolic links works in the same manner as do hard links: *ln -s SOURCE LINK* The primary difference between hard and symbolic link creation, is that you use the *-s* option. Let's create a symbolic link from *\~/file2* to */data/file2* in similar fashion as we did above, only we'll create a symbolic link, instead of a hard link. Here's how that would be accomplished: *ln -s /data/file2 \~/file2* The above command will create a symbolic link from \~/file2 to the original location */data/file2*. If you update the file in either location, it will update in both. It is also important to note that you can use symbolic links for directories. Say, for instance, you have */data/directory1* and you want to create a symbolic link to that directory in *\~data/directory1*. You can then add to that directory from either location and the change will reflect in both. To see the difference between how each type of link looks from a terminal window, issue the command ls -li. You will see how each is represented with slight variation from one another (Figure 1). **UNIT 6** **SEARCHING FOR FILES IN LINUX SYSTEM** In this unit, we will introduce two useful tools to search for files within the Linux filesystem. The first one is the find command, which can be found in any Linux distribution, and the second one is locate, which needs an installation. ### **Find Command** The find command is part of Linux by default. It offers multiple parameters to refine your search. #### **Find by Name** The simplest searching command is to search by name where you are located: \$ find -name \"FILE\_NAME\" You can specify the directory where you want to search for the file: *\$ find /home -name my\_file.txt* *\$ find / -name my\_other\_file.txt* Take into consideration that this command is case sensitive. If you want to ignore the case use: *\$ find -iname \"FILE\_NAME\"* #### #### Inverse Search In case you want to search for files that do not adhere to the specified pattern you can negate the search parameters by using the -not parameter or \\!. For example: *\$ find -not -name \"FILE\_NAME\_TO\_AVOID\"* *or* *\$ find \\! -name \"FILE\_NAME\_TO\_AVOID\"* #### #### **Find by File Type** The file command also allows you to search for file types. Some common file type descriptors are: - f: regular file - d: directory - l: symbol link - c: character devices - b: block devices - The desired file type goes after the -type attribute. For example, to find al JSON files under the etc directory: *\$ find /etc -type f -name \"\*.json\"* #### #### **Find by File Size** There is also a parameter used to filter the search result depending on the size of the files: -size. For the size units you can use the following suffixes: - c: bytes - k: kilobytes - M: megabytes - G: gigabytes - b: 512-byte blocks And to specify if you are talking about an exact size, a less than, or greater than use the following prefixes: - *Exact search*: no prefix - *Less than*: - prefix - *Greater than*: + prefix For example, to find all 3 gigabytes files under the home directory: *\$ find /home -size 3G* *For all files under 3 gigabytes:* *\$ find /home -size -3G* *Files over 3 gigabytes:* *\$ find /home -size +3G* #### #### **Search by Time** You can also search files according to their last access, modification, or change times. - *Access time: -atime for the last time the file was read.* - *Modification time: -mtime for the last time the file contents were modified.* - *Change time: -ctime for the last time where the file\'s inode meta-data was changed.* It is also possible to compare against a reference file to return the ones that are newer: *\$ find / -newer file\_name* For the times, the parameters specified are in **days**. The same prefixes as in search by size can be used: *\$ find /home -mtime 3* *\$ find /home -atime -2* *\$ find /home -ctime +5* **To find files using time in minutes, you can use:** *\$ find /home -mmin 3* #### #### **Find by Owner and Permissions** There are also parameters to find files according to their owners (-user and -group) and their permissions (-perm): *\$ find /home -user my\_user* *\$ find /home -group my\_group* *\$ find /home -perm 777* #### #### **Defining Search Depth** When searching under a directory, the find command will return all files under that directory, even if they are inside other subdirectories. To control this depth you can define a maximum depth, a minimum depth, or both. For example, to search only on the top directory and one level of subdirectories you could set the -maxdepth parameter to 2: *\$ find -maxdepth 2 -name my\_file* *To define the minimum depth where find should search you use the -mindepth parameter:* *\$ find -mindepth 5 -name my\_file* *These parameters can be combined:* *\$ find -mindepth 4 -maxdepth 7 -name my\_file* #### #### **Executing Commands on Results** As saw in the depth commands, all find parameters can be combined to obtain very specific results. When you get the results you wanted it is possible to execute commands over them by using the -exec parameter. When combining search parameters you can also use the -and and -or options to personalize your search criteria. When none of them are used, the -and option is the default. For example, to change the permissions to all files under the /my\_dir directory that currently have 777 permissions to 664 use the following: *\$ cd /my\_dir* *\$ find. -type f -perm 777 -exec chmod 664 {} \\;* ### ### **Locate Tool** The locate command is an alternative for find. The difference is that it uses a database of the files in the filesystem so it can perform quicker. To install the locate tool you can use: - *\$ sudo apt install mlocate\`* - *\$ sudo yum install mlocate\`* *Locate uses a cron job to update the database daily, but it can be manually updated by using:* *\$ sudo updatedb* For searching the \"basename\", namely that the query is contained only in the file\'s name and not in the file\'s path, use the -b flag: *\$ locate -b name* *To find files that still exist (after the last updatedb call) use the -e flag:* *\$ locate -e name* *For statistics from the generated database use:* *\$ locate --S* ### In Conclusion, Searching for files in the Linux command line might be difficult. The tools presented in this article provide an easy way to handle and find files. By understanding the different parameters the search can be as specific as one want it to be. **UNIT 7** **WORKING WITH USERS AND GROUPS IN LINUX SYSTEM** On Linux, group information is held in the /etc/group file. You can use commands to create a group, add a user to a group, display a list of the users who are in the group, and remove a user from a group. Procedures ---------- 1. To create a new group, use the groupadd command. Type the following command: *groupadd -g group-ID* *group-name* Where *group-ID* is the numeric identifier of the group, and *group-name* is the name of the group. 2. To add a member to a supplementary group, use the usermod command to list the supplementary groups that the user is currently a member of, and the supplementary groups that the user is to become a member of. *usermod -G groupa,groupb user-name* Where *user-name* is the user name. 3. To display who is a member of a group, use the getent command. Type the following command: *getent group group-name* Where *group-name* is the name of the group. 4. To remove a member from a supplementary group, use the usermod command to list the supplementary groups that you want the user to remain a member of. For example, if the user\'s primary group is users and the user is also a member of the groups mqm, groupa and groupb, to remove the user from the mqm group, use the following command: *usermod -G groupa,groupb user-name* Where *user-name* is the name of the user. **UNIT 8** **WORKING WITH FILE PERMISSIONS IN LINUX SYSTEM** All the three owners (user owner, group, others) in the Linux system have three types of permissions defined. Nine characters denotes the three types of permissions. 1. **Read (r):** The read permission allows you to open and read the content of a file. But you can\'t do any editing or modification in the file. 2. **Write (w):** The write permission allows you to edit, remove or rename a file. For instance, if a file is present in a directory, and write permission is set on the file but not on the directory, then you can edit the content of the file but can\'t remove, or rename it. 3. **Execute (x):** In UNIX type system, you can\'t run or execute a program unless execute permission is set. But in Windows, there is no such permission available. **Permissions are listed below:** **permission** **on a file** **On a directory** ---------------- -------------------------- ---------------------------------- r (read) read file content (cat) read directory content (ls) w (write) change file content (vi) create file in directory (touch) x (execute) execute the file enter the directory (cd) **Ten Characters (-Rw-Rw-R\--) Before the User Owner** **File Permissions for (-rw-rw-r\--)** **position** **characters** **ownership** -------------- ---------------- ---------------------- 1 \- denotes file type 2-4 rw- permission for user 5-7 rw- permission for group 8-10 r\-- permission for other When you are the **User owner**, then the user owner permission applies to you. Other permissions are not relevant to you. When you are the **Group** then the group permission applies to you. Other permissions are not relevant to you. When you are the **other,** then the other permission applies to you. User and group permissions are not relevant to you. First letter (-) or **d** represents the files and directories respectively. Now, from remaining nine letters, first triplet represents the permission for user owner. ----------------------------------------------------------------------------------------- Second triplet represents the permission for group owner. --------------------------------------------------------- Third triplet represents the permission for other. Setting Permissions with chmod --------------------------------------------------------------------------------- You can change the permissions with chmod command accordingly to your need. Below are some examples to change the permissions for different groups. 1. **Syntax:** chmod \10 && NR\