Scientific Computing and Linux

• This lecture introduces scientific computing and Linux, and focuses on the Linux command line.
• MATH 457 introduces scientific computing and Linux.
• The presenter is Elliot S. Menkah, Ph.D. who is from the National Institute for Mathematical Sciences, Ghana; Kwame Nkrumah University of Science and Technology, Ghana; and ICTP, Trieste.

Scientific Computing and Linux

• The use of computers to analyze and solve problems such as automating large matrix vector operations is scientific computing.
• It enables study of mathematical models of physical phenomena.
• It helps to identify optimal system parameters.
• Computers are used by experimentalists to control experiments and gather relevant data in scientific computing.

Linux Command Line Outline

• The course will cover the file system, including the Linux file system.
• Focus will be on basic commands, file operations, user environment, access control, process management, and network management.
• The vim text editor will be introduced.
• Shell tools and programming, including bash scripting, variables, statements, conditionals, control sequences/loops, and functions will be covered.
• Regular expressions will be introduced.

File System

• Filesystem types include ext2, ext3, ext4, reiserfs, vfat, xfs, and nfs.
• Devices include block devices and loop devices.
• Topics include inodes, Filesystem Hierarchy Standard (FHS), and Network File System (NFS).
• Basic file operations and the Linux directory structure will be reviewed.

Basic Operations

• Basic commands
• File operations
• User environment
• Access control
• Process management
• Network management

Basic Commands

• ls: Lists files
• pwd: Shows the present working directory.
• cd: Change directory with cd directoryname.
• cat: Lists file content
• alias: Remaps a command.
• date: Checks or sets the date/time.
• uname: Provides OS information, version, and architecture.
• touch: Creates new files with touch filename.
• mkdir: Creates a new directory with mkdir directoryname.
• cp: Copies files and directories with cp file-to-be-copied new-destination.
• To copy a directory recursively: cp -r directory-to-be-copied new-destination.
• mv: Relocates or renames a file/directory with mv file-to-be-moved new-destination.
• rm: Deletes files and directories.

Process Management

• ps: Lists processes.
• kill: Kills processes.
• top: Monitors processes.
• fuser: Finds the process owner.

User Environment

• env: View user environment
• export: add to the user environment.
• bashrc: stores user environment settings.
• profile: store global user environment settings
• tilde: reference to the current user.

File Operations

• echo: Displays lines of text or strings.
• grep: Matches string patterns in text.
• paste: Joins content of files horizontally.
• cut: Cuts out sections of a line of text.
• file: Provides file information.
• find: Finds files matching specific criteria.
• xargs: Parses arguments.
• tar: Archives and uncompresses files.

Shell Tools: Echo

• ECHO is a command-line tool displays lines of text or strings passed as arguments.
• Mostly used to output status text to the screen or a file.
• Structure is echo [options] string.
• Options:
  • -e: Allows you to change the format of text.
  • -n: Removes preceding newline.
• Escape characters:
  • \a: Audible alert
  • \b: Backspaces character just before the slash
  • \c: Truncates everything after the slash.
  • \n: Adds a new-line character.
  • \t: Adds a tab character to the output.
• > redirects output to a new file, overwriting existing content.
• >> redirects output and appends to a file.
• cat file-name command verifies the content of a file.

Shell Tools: Grep

• GREP searches plain-text data sets for lines that match a regular expression.
• Used for line matching and extraction.
• Supports regular expressions and inverse matching (-v).
• It supports piping.
• Structure: grep [options] pattern-being-sort [files]
• Options:
  • -w: Match exact words.
  • -n: Provide lines of occurrence.
  • -i: Case-insensitive pattern.
  • -r: Recursive search and match.
  • -c: Count.
  • -A: Lines After context.
  • -B: Lines Before context.
  • -C: Lines Before & After context.

Shell Tools: Cut

• CUT is a command-line utility that cuts out sections of string of text.
• It cuts certain section of line from files, at byte positions, characters or fields.
• Structure: cut [options]... [FILES] ...
• Options:
  • -b: Extract by bytes
  • -c: Extract by character
  • -f: Extract by fields
• The -f option uses a tab space as the default delimiter which is denoted by -d and can be changed

Shell Tools: Paste

• PASTE is a command-line utility joins files horizontally (parallel merging) by outputting lines consisting of lines from each file specified using tab as the delimiter.
• Structure: paste [options]... [FILES] ...
• Options:
  • -d: Delimiter
  • -s: Sequential merging

Practical Tools

• Command: find . -type f - xargs grep elliot

Shell Tools: AWK

• AWK is a command-line utility for tasks such as text processing and data extraction.
• It facilitates data manipulation and report generation.
• It filters and cn scans files line by line
• Splits the input lines into fields and can compare lines/fields to patterns.
• If matches are found perform actions
• Syntax: awk [options] 'selection_criteria action' input-file > output
• Command: awk '{ print $1}' data_output.dat > log0.txt extracts the first column.
• To extract data you can redirect the output of one command as the input of awk
• Command: awk -F ':' '{ print $1 }' > users.txt extracts usernames from /etc/passwd, with ':' as the delimiter.
• Field separators for both delimeer field and Output Field can be predefined
• To separate the output by tab spaces using ":" use the command: awk 'BEGIN{FS=":"; OFS="\t"} { print $1, $3}' > users.txt
• To get names starting with "ic" from /etc/passwd use the command: awk 'BEGIN{FS=":"; OFS="\t"} /^ic/ { print $1, $3}' > users.txt
• To divide all values of column 1 by 2.0 use the command: awk '{print $1/2.0 }' data_output.dat
• To extract the running processes with bash names starting use the command: ps -ef | awk '{ if($NF == "bash") print $0 }'

Shell Tools: Sed

• SED (Stream Editor) used for parsing and transforming text.
• This tool performs line stream matching and extraction
• input is file
• Supports regular expressions and piping.
• Syntax: sed [options] [SCRIPT] input-file > output
• To replace the string Kinetic in data_output.dat to Total use the command: sed s/Kinetic/Total/ data_output.dat

Text Editing and Shell Scripting

• Editing with Linux text editors: Nano, Vi or Vim, and Emacs.
• Bash Shell Scripting

Nano

• The syntax to start text editing with Nano is nano <file-name>.
• Editing operations:
  • ctrl + O: Write to file (save changes).
  • ctrl + X: Close the opened file.
  • ctrl + G: Get help.
  • ctrl + W: Search or find a string.

Vim

• Command to start text editing with Vi or vim is vim <file-name>.
• The escape mode is activated with the esc key
• Modes of Vi/Vim: Escape mode (esc key), INSERT mode (i key), and VISUAL Block mode (ctrl + v).
• Editing operations when in escape mode are that w will write to file (save changes made), and q will quit vim of close the opened file

Shell Scripting

• Shells : Borne Shell, Borne-Again Shell(Bash), korn shell, C shell.
• Importance of shell scripting: Writing a series of commands, combining lengthy and repetitive commands, and executing routine tasks.
• Steps to create a shell script:
  • Create a file with a .sh extension via text editor.
  • Start the script with a shebang: #!/path/to/shell/.
  • Add code/text/content to the file/script and save.
  • Modify file permissions to make it executable.
• chown is used to change ownership of files.
• chmod is used to change permission on files.
• setuid is used to to Share ownership on files
• sticky bit: Share write access on a directory
• Set the permission for a file to be executable by the user by executing the command chmod u+a <script-name.sh>.
• The command ./<script-name.sh> or bash <script-name.sh> are used to run or execute a file

Basic Concepts in Shell Scripting

• Topics covered: Comments, Variables, Statements, Conditionals, Controls sequence/Loops and Functions
• Comments in shell scripting start with #.
• Conditionals are tools for decision making.
• Control Sequence or loops are used to iteratively parse instructions to be executed.
• Functions provide a way or technique for grouping reusable bits of code under one name for later use.