Bash Command Basics

Users interact with the Grid using a text-based interface. Because the Grid is text-based users need to know some shell commands in order to run their programs (shell refers to the user interface for text-based interfaces such as the command prompt in UNIX). This guide covers Bourne Again Shell (Bash) commands, but there are other types of shells you can learn as well (such as C shell). Bash is a programming language so there is a lot you can learn with it. However, to run jobs on the Grid you only need to know some shell programming, you do not need to be an expert. This short guide covers basic bash commands, text editors you can use such as Vim, and instructions for compiling and linking code on the Grid using the make built-in shell command.

Commands

Users interact with the Grid using the shell interface. There are Sun Grid Engine (SGE) specific commands and shell commands. This guide covers the Bourne Again Shell (Bash) variation of shell commands. With Bash you can learn countless commands. You need to know some basic shell commands to run jobs on the Grid. Below are some useful Bash commands, many of which deal with file and directory manipulation. Pay attention to use of upper and lowercase letters because shell commands, as well as file and directory names, are case sensitive. You can look up a description of a command by typing man (which stands for 'manual') and then the command. For example, man ls shows information about the commands ls.

Files and Directories

Here are some common and useful bash commands, most of which deal with file and directory manipulation:

  • ls: Show directory contents, lists names of files.
  • mkdir: Creates a directory of the specified name.
    • mkdir foo creates a directory called "foo".
  • cat: Display contents of a file.
  • cd: Change directory. Change to certain directory name if provided.
    • cd foo takes you to the directory foo.
    • Changes to home directory if no directory specified.
  • pwd: Displays the name of the working directory.
  • touch: Creates a blank file with a specified name.
  • less: View contents of specified file, page by page.
  • head/tail: Displays the first/ last 10 lines of a file.
  • rm: Removes a specified file. This action is permanent. There is no recycle bin.
  • rmdir: Removes a directory.
  • history: Display a listing of the last commands you've run.
  • cp: Copy specified file to a new named file. Use -r flag to copy a directory.
  • mv: Rename a specified file or directory.
  • find: search files and directories. Can use with wildcards (* ? [ ]).
  • quota: Print the amount of space available and used on all shares for the current user.
  • scp: Secure/ SSH copy. Copies from either the local filesystem to a remote filesystem, or vice versa. This will not work if both arguments are on remote systems. The remote system must be specified as user@remote.fqdn:/where/to/put. The user part is optional and only needed if the remote account name doesn’t match the local one. remote.fqdn is the remote machine, the colon afterwards tells scp that you’re giving it a path, and the path tells it where to place the file.

Control Structures

All computer programming languages use control structures, which are instructions about how to carry out steps. Here are some sample control structure syntaxes in Bash.

if...then...else

if [command]
then
commands
else
commands
fi - opposite of if. It signifies the end of the if...then statement.

for...in

for loop-index in argument
do
commands
done

while

while [test-command]
do
commands
done

Input/ Output Redirection

Bash uses three main streams of data: input, output and error. You can direct input and output of data with various commands.

  • < : Takes the contents out of the file and redirects it to the program.
  • > : Takes the STDOUT (output stream) of the specified program, and writes it to a certain file. Any contents already in the file are overwritten.
  • >> : Takes the STDOUT of the specified program and appends (adds) it to the specified program. The original contents of the file remain, instead of being deleted.
  • | : Takes the output of a specified program and redirects it as input to another program (called piping).

Compression

Files can be compressed and extracted using the commands tar, gzip/gunzip, and lzma/unlzma. You can change how the compression works by using various modifiers on the commands.

  • tar: Short for Tape Archiver is an analogue to zip under Linux.
    • When run as tar -cf foo.tar bar the file or directory (tar doesn't care) called "bar" will be put into the archive foo.tar.
    • When run as tar xf foo.tarit will extract the archive.
      • The extract command can be specified with -xf or just as xf.
      • If the "-" is omitted, tar will intelligently figure out what, if any compression (except for lzma), was used on the tarball and uncompress it.
  • gzip/gunzip: Run gzip compression on a file.
    • This method is fast, but the compression ratio (how small the file is when compressed compared to the original size) is not as good as lzma.
  • lzma/unlzma: Run LZMA compression on a file.
    • This method is slower, but has the best compression ratio in most, but not all, cases.

Wildcards

Wildcards act as substitutions for character strings. They can be used with search and other functions.

  • *: zero or more characters.
  • ?: any one character
  • [ ]: any character within the brackets

Example: rm *.o would remove all files that ended with .o in a directory.

Text Editors

To create Bash scripts many people use text editors. Vim and Emacs are two common and popular editors. Both require a lot of knowledge to operate well. Here is a brief overview of how to use Vim, and some information about Emacs.

Vim

Vim is a text editor commonly used for creating shell scripts. Vim has many features which make creating and editing shell scripts easier. Users can use many commands to control their scripting and there is a lot to learn with vim.

Different modes

Vim has three main "modes" of functioning: command mode, input mode, and last line mode. Users alternate between inputting text and executing commands. Press i to enter Input mode. Press Esc to enter command mode. Press : to enter last line mode.

Command mode

In command mode users type text to execute built-in vim commands.  Press Esc to enter command mode. Commands will happen automatically after they are typed. There are countless commands to be used in Vim. Here is a sample of some basic commands:

  • Esc key: enter command mode
  • f or F: search current line for specified character
  • ZZ: save file and exit
  • y or Y: yank file (similar to copy function)
  • p or P: put file (similar to paste function)
  • dd or dw: delete
  • u: undo
  • /: search document for string of text
Input mode

Use input mode to type text. Press i to enter input mode. You can navigate the document using the arrow keys.

Last Line Mode

You can execute special commands using last line mode. Press : to enter it. For example, type : help to access a built-in help document. Another example, type :%s/search/replace/ to replace the first occurrence of the string "search" with the string "replace". You can also add a "g" after the third "/" to replace for the whole file. There are many other features of last line mode as well.

Emacs

Emacs is another popular text editor. Instead of using modes like vim it uses keystroke combinations and uses the ctrl key for commands. Many tutorials for Emacs can be found online.