Overview

If you are going to use bash shell scripting to create programs that will manipulate computer data, then it is useful to control what data is input to a command, control where the data is output, and also control where error messages are output.

File Descriptors

In the bash shell, command input and command output can be manipulated. For every command that can be used in bash there are three file descriptors: standard input (stdin), standard output (stdout), and standard error (stderr).

• Standard input - is the information that is passed to a command. The standard input information can be input by the user through the blinking prompt, or it can be passed to a command from a file or program. Standard input is also represented by the number 0 (see below)

• Standard output - is the data output after a command has executed. Standard output is also represented by the number 1 (see below)

• Standard error - are any error messages that may have been generated by a command. Standard error is also represented by the number 2 (see below)

Redirectors

In bash, standard input, standard output and standard error messages are manipulated by redirect command characters. Redirects control how data is input and output from files to commands and vice versa:

• The output redirector (> or 1>) redirects standard output to a file instead of the screen. Example:
  $ ls -l /var > mydirectory.txt
  $ ls -l /var 1> mydirectory.txt
• The input redirector (< or 0<) redirects standard input from a file. Example:
  $ ls -l < /var
  $ ls -l 0< /var
  $ cat < mydirectory.txt
  $ cat 0< mydirectory.txt
• The append redirector (>> or 1>>) appends standard output to the end of a file instead of rewriting it. Example:
  $ ls -l Documents > myfiles.txt
  $ ls -l Downloads >> myfiles.txt
  $ ls -l Pictures 1>> myfiles.txt
• The output redirector with a 2 (2>) redirects standard error to a file. Example:
  $ ls -l Pixxtures 2> errors.txt
  
• The output redirector with a 1 and a 2 (1> 2>) can be use to redirect standard output and standard error to two different files. The number one in the first redirector is not necessary since a greater-than sign by itself implies the number one, standard output. Example:
  $ ls -l Documents Pixxtures > good.txt 2> errors.txt

Pipes

The pipe is a special command character (|) which can take the standard output (stdout) from one command and make it the standard input (stdin) for another command. In the example below, the command ls -l has been issued to list the contents of the /etc directory, but before the output can be sent to the screen, it is piped (|) and turned into standard input for the more command to process. The more command takes the input and executes its program, which processes the information and outputs it to the display it one screen at a time.

$ ls -l /etc | more

In this next example, the pipe is used to take the data output from the ls -l /etc command and send it as input to the grep command to search for a specific lines of text that have the text “firefox” in them.

$ ls -l /etc | grep “firefox”

Pipes are very useful because they allow you to send information from one command to another in sequence. Pipes and redirectors can be used together very powerfully to control how data is input and output.

Video Tutorials

In this tutorial, I demonstrate using redirector characters to control standard input, output, and error

In this video, I demonstrate how to use of the pipe special character and the
grep command, to filter output to find specific lines of content

Overview of Bash Shell Scripting

The default Linux command line interface or terminal is the Bourne Again shell or bash shell. A shell is a user interface to a computer system that relies on keyboard input, as opposed to a graphical user interface which relies on keyboard input, mouse input, and presents the user with graphical icons and windows to click on with a mouse.

Since bash is a command interpreter, it has powerful programming capabilities like a full fledged programming language. In addition to entering commands in the terminal one at a time, you can write commands into a text file, save it, and execute it like a program. An executable file with shell commands is called a shell script or shell program.

To make a shell script executable, you have to give the text file, execute permissions. You do this with the chmod command:

$ chmod + x <filename>

If you try to run a shell script that does not have execute permissions you will get a Permission denied message. You can start by learning how to create a simple “Hello World!” shell script.

A “hello world!” shell script

1. Start by opening a Linux terminal.

2. In the terminal, create the shell script file using a text editor like Nano. If you like, you can name the file with a .sh file extension for visual reference that it is a shell script. The file extension is optional, since Linux does not rely on file extensions like Windows to determine the file type. The command structure: $ nano <filename>

$ nano hello.sh

3. With nano open you can type your first shell script. The first line in the script starts with a hash-bang (#!) followed by the path to the bash program. This lets the terminal know that it is bash specifically, that you want to use to execute this script. This is important since there are other terminal shells, like the korn shell, c-shell, the tcshell, etc.. Type the following text into nano, then do control+x on the keyboard, type y for yes, and press enter, and then press enter again to accept the file name.

#!/bin/bash
echo “Hello world!”

4. Now that you have the file saved, do a cat command to verify that the text saved to the file. If it was saved, you will see the text of hello world returned. The command structure: $ cat <filename>

$ cat hello.sh
Hello World!

5. Now you need give the file execute permissions. The command structure: $ chmod +x <filename>

$ chmod +x hello.sh

6. Now do a ls -l command to see if execute permissions were applied. The command structure: $ ls -l <filename>

$ ls -l hello.sh

The output should look like this. If the line leads with a “d” it means it is a directory and if it is a “-” it means it is a file. Notice that after the file character “-“, the permissions on the hello.sh line read rwxr-xr-x, indicating that the owner is read, write, execute (rwx), followed by the group permission of read and execute (r-x), and the public, or everyone, which is also read and execute (r-x). The execute bit was successfully applied to all three groups: owner, group, and everyone.

drwxr-xr-x 2 dan dan 4096 Jan 12 16:17 Desktop
drwxr-xr-x 4 dan dan 4096 Jan 17 19:53 Documents
drwxr-xr-x 3 dan dan 4096 Jan 16 07:47 Downloads
-rwxr-xr-x 1 dan dan 43 Jan 24 19:25 hello.sh

7. Now that your shell script file has execute permissions you can run it as a program. Since the file is not saved in a directory that is included in the $PATH variable you cannot run it like a program. You can verify this by simple trying to run your shell script by typing $ hello.sh you should see the command not found returned. However, you can run your program by giving the absolute path to the file or referencing the current directory with a “./”.

$ /home/user/hello.sh
Hello world!
or simply,
$ ./hello.sh
Hello world!

8. You have the basics on creating and running a shell script in the bash shell. Now, expand your knowledge by learning additional programmatic capabilities that can be used to write more advanced scripts that will automate basic to advanced system tasks.

A shell script to backup your home directory

1. Open a Linux terminal.

2. Create the file in nano

$ nano backup.sh

3. Start typing your shell script. This time you can use the echo command to explicitly output what it is the script is attempting to do. You will use the tar command to create a compressed file of your home folder and save it to the var directory. The command structure is: $ tar -czf <destination-directory> <file-to-backup>. After the tar command, you can use the echo command to notify the user that the backup process is complete. Control+x and save the file. You will need to change /home/dan to the path to your home directory.

#!/bin/bash
echo “Backing up home directory”
tar -czf /var/homebackup.tgz /home/dan
echo “Done!”

4. Do a cat command to verify that the file was saved correctly. The command structure: $ cat <filename>

$ cat backup.sh

5. Give the file execute permissions. The command structure: $ chmod +x <filename>

$ chmod +x backup.sh

6. Now do a ls -l command to see if execute permissions were applied. The command structure: $ ls -l <filename>

$ ls -l backup.sh

7. Now that your shell script file has execute permissions, go ahead and run it. You will notice that the script fails because you do not have super user permission to save to the /var directory. Run the script again with a sudo command in front, enter your password, and you will see that the shell script file executes successfully.

$ ./backup.sh
Backing up home directory
tar: Removing leading `/’ from member names
tar (child): /var/homebackup.tgz: Cannot open: Permission denied
tar (child): Error is not recoverable: exiting now
Done!

$ sudo ./backup.sh
Backing up home directory
tar: Removing leading `/’ from member names
Done!

Video Tutorials

In this video, I write a basic shell script, give it execute permissions, and run it as a program

In part 2, I write a basic shell script to backup the user’s home directory