bash-handbook

This document was written for those who want to learn Bash without diving in too deeply.

Node Packaged Manuscript

You can install this handbook with npm. Just run:

$ npm install -g bash-handbook

Now you should be able to run bash-handbook as command that will open this README file in your $PAGER. Otherwise, you may continue reading this document.

Introduction
Shells and modes
Comments
Variables
Arrays
Shell expansions
Streams, pipes and lists
Conditional statements
Loops
Functions
Debugging
Afterwords
License

Introduction

If you are a developer, then you know the value of time. Optimizing your work process is one of the most important aspects of the job.

In that path towards efficiency and productivity, we are often posed with actions that must be repeated over and over again, like taking a screenshot and uploading them to a server, processing text that may come in many shapes and forms, converting files between different formats, parsing a program's output and the list goes on. Enter bash, our savior.

Bash is a Unix shell written by Brian Fox for the GNU Project as a free software replacement for the Bourne shell. It was released in 1989 and has been distributed as the Linux and OS X default shell for a long time.

So why do we need to learn something that was written more than 30 years ago? The answer is simple: this something is today one of the most powerful and portable tools for writing efficient scripts for all Unix-based systems. And that's why you should learn bash. Period.

In this handbook, I'm going to describe the most important concepts in bash with examples. I hope this compendium will be helpful to you.

Shells and modes

The user bash shell can work in two modes - interactive and non-interactive.

Interactive mode

If you are working on Ubuntu, you can open the shell using the Ctrl-Alt-F1 keybinding. After that, the familiar GUI will disappear and one of the seven virtual terminals available in Ubuntu will be shown.

If you see something like this, then you are working in interactive mode:

user@host:~$

Here you can enter a variety of Unix commands, such as ls, grep, cd, mkdir, rm and see the result of their execution.

We call this shell interactive because it interacts directly with the user.

The desktop environment takes place in the seventh virtual terminal, so you can return to friendly GUI using the Ctrl-Alt-F7 keybinding.

Using a virtual terminal it's not really convenient. For example, if you want to edit a document and execute another command at the same you are better off using a virtual terminal emulators like:

Non-interactive mode

In non-interactive mode, the shell reads commands from a file or a pipe and executes them. When the interpreter reaches the end of the file, the shell process terminates the session and returns to the parent process.

Use the following commands for running the shell in non-interactive mode:

sh /path/to/script.sh
bash /path/to/script.sh

In the example above, script.sh is just a regular text file that consists of commands the shell interpreter can evaluate and sh or bash is the shell's interpreter program. You can create script.sh using your preferred text editor (e.g. vim, nano, Sublime Text, Atom, etc).

You can also simplify invoking the script by making it an executable file using the chmod command:

chmod +x /path/to/script.sh

Additionally, the first line in the script must indicate the system which program it should use to run the file, like so:

#!/bin/bash
echo "Hello, world!"

Or if you prefer to use sh instead of bash, change #!/bin/bash to #!/bin/sh. This #! character sequence is known as the shebang. Now you can run script like this:

/path/to/script.sh

Another handy thing we learned above is using the echo to print something to the terminal screen.

Exit codes

Every command returns an exit code (return status or exit status). A successful command always returns 0 (zero-code), and a command that has failed returns non-zero value (error code). Failure codes must be positive integers between 1 and 255.

Another handy command we can use when writing a script is exit. This command is used to terminate the current execution and deliver an exit code to the shell. Running exit code without any arguments, will terminate the running script and return the exit code of the last command executed before exit.

When a program terminates, the shell assigns its exit code to the #? environment variable. The #? variable is how we usually test whether a script has a succeeded or not in its execution.

In the same way we can use exit to terminate a script, we can use the return command to exit a function and return an exit code to the caller. You can use exit inside a function too and this will exit the function and terminate the program.

Comments

Scripts may contain comments. Comments are special statements ignored by the shell interpreter. They begin with a # symbol and continue on to the end of the line.

For example:

#!/bin/bash
# This script will print your username.
whoami

Tip: Use comments to explain what your script does and why.

Variables

Like in most programming languages, you can also create variables in bash.

Bash knows no data types. Variables can contain only numbers or a string of one or more characters. There are three kinds of variables you can create: local variables, environment variables and variables as positional arguments.

Local variables

Local variables are variables that exist only within a single script. They are inaccessible to other programs and scripts.

A local variable can be declared using = sign (as a rule, there should not be any spaces between a variable's name, = and its value) and its value can be retrieved using the $ sign. For example:

username="denysdovhan"  # declare variable
echo $username          # display value
unset username          # delete variable

We can also declare a variable local to a single function using the local keyword. Doing so causes the variable to dissapoear when the function exits.

local local_var="I'm a local value"

Environment variables

Environment variables are variables accessible to any program or script running in current shell session. They are created just like local variables, but using the keyword export instead.

export GLOBAL_VAR="I'm a global variable"

There are a lot of global variables in bash. You will meet these variables fairly often, so here is a quick lookup table with the most practical ones:

Variable	Description
`$HOME`	The current user's home directory.
`$PATH`	A colon-separated list of directories in which the shell looks for commands.
`$PWD`	The current working directory.
`$RANDOM`	Random integer between 0 and 32767.
`$UID`	The numeric, real user ID of the current user.
`$PS1`	The primary prompt string.
`$PS2`	The secondary prompt string.

Follow this link to see an extended list of environment variables in Bash.

Positional arguments

Positional arguments are variables allocated when a function is evaluated and are given positionally. The following table lists positional argument variables and other special variables and their meanings when you are inside a function.

Parameter	Description
`$0`	Script's name.
`$1 … $9`	The argument list elements from 1 to 9.
`${10} … ${N}`	The argument list elements from 10 to N.
`$*` or `$@`	All positional parameters except `$0`.
`$#`	The number of arguments, not counting `$0`.
`$FUNCNAME`	The function name (has a value only inside a function).

In the example below, the positional arguments will be $0='./script.sh', $1='foo' and $2='bar':

./script.sh foo bar

Variables may also have default values. We can define as such using the following syntax:

 # if variables are empty, assign then default values
: ${VAR:='default'}
: ${$1:='first'}

Arrays

As in other programming languages, an array is a variable containing multiple values. In bash arrays are zero-based: that means the first element in arrays is indexed with the number 0.

Dealing with arrays, we should know about special environment variable IFS. IFS or Input Field Separator — it's value that contains the character which separates elements in array. As default IFS=' '.

Array declaration

It's easy to declare arrays using these indirect declaration:

fruits[0]=Apple
fruits[1]=Pear
fruits[2]=Plum
echo ${fruits[*]} # echo ${fruits[@]} may be used as well

Array variables may also be created using compound assignments such as:

fruits=(Apple Pear Plum)

Array slice

Besides, we can take part of array using slice:

echo ${fruits[*]:0:2} # Apple Pear

In example above, fruits[*] returns the content of array, :0:2 takes a slice length of 2, starting at index 0.

Adding elements into array

If we wanna add new elements into array, we are happy, because it quite simple. Compound assignments come to help us here. We can use them such as:

fruits=(Orange ${fruits[*]} Banana Cherry)
echo ${fruits[*]} # Orange Apple Pear Plum Banana Charry

Deleting elements from array

The unset command, which we are already familiar, is used to destroy arrays or element of array:

unset fruits[0]
echo ${fruits[*]} #

Shell expansions

Expansion is performed on the command line after it has been split into tokens. In other words, these expansions are mechanism to calculate arithmetical operations, to save results of command's executions and so on.

If you are interested, you can read more about shell expasions.

Brace expansion

Brace expansion give us opportunity to generate arbitrary strings. It's similar to filename expansion. For example:

echo beg{i,a,u}n # begin began begun

Also brace expansions may be used for creating ranges, which are iterated over in loops.

echo {0..5} # 0 1 2 3 4 5
echo {00..8..2} # 00 02 04 06 08

Command substitution

Command substitution allows the output of a command to replace the command itself. Command substitution works when a command is enclosed in `` or $(). For example, we can use it as follow:

now=`date +%T`
# or
now=$(date +%T)

echo now # 19:08:26

Arithmetic expansion

In bash we can feel free to perform arithmetical operations. But expression that we need to perform should be enclosed in $(( )) The format for arithmetic expansion is:

result=$(( ((10 + 5*3) - 7) / 2 ))
echo $result # 9

Double and single quotes

There is an important difference between double and single quotes. The expression is treated as if it were within double quotes, but a double quote inside the parentheses is not treated specially. Just look at this:

echo "Your home: $HOME" # Your home: /Users/<username>
echo 'Your home: $HOME' # Your home: $HOME

Streams, pipes and lists

Bash has powerful tools for working with other programs and theirs outputs. Thanks for streams we can send outputs of programs into files and thereby write log or whatever you want.

Pipes give us opportunity to create conveyors and control the execution of commands.

Undoubtedly, we should know how to use this high-powered tool.

Streams

Bash receives input and sends output as sequences or streams of characters. These streams may be redirected into files or one into another.

There are three descriptors:

Code	Descriptor	Description
`0`	`stdin`	The standard input.
`1`	`stdout`	The standard output.
`2`	`stderr`	The errors output.

Redirection makes it possible to control where the output of a command goes to, and where the input of a command comes from. For redirecting streams these operators are used:

Operator	Description
`>`	Redirecting output
`&>`	Redirecting output and error output
`&>>`	Appending redirected output and error output
`<`	Redirecting input
`<`	Redirecting input
`<<`	Here documents syntax
`<<<`	Here strings

Here are few examples of using redirections:

# output of ls will be written to a file
ls -l > list.txt

# append output to a file
ls -a >> list.txt

# all errors will be written to a file
grep da * 2> errors.txt

# read from file
less < errors.txt

Pipes

We could redirect standard streams not only in files, but also to other programs. Pipes let us use the output of a program as the input of another one.

Below, command1 send its output to command2, which send its output to the input of command3:

command1 | command2 | command3

Constructions like this are called pipelines.

In real world it can be used for processing data through few programs. For example, here the output of ls -l is sent to the grep program, which will print only files with .md extension, and after all, output will be sent to the less program:

ls -l | grep .md$ | less

Lists of commands

A list of commands is a sequence of one or more pipelines separated by ;, &, && or || operator.

If a command is terminated by the control operator &, the shell executes the command asynchronously in a subshell. In other words, this command will be executing in background.

Commands separated by a ; are executed sequentially: one after another. The shell waits for finish of each command.

# command2 will be executed after command1
command1 ; command2

Lists separated by && and || are called AND and OR lists, respectively.

The AND-list looks like this:

# command2 will be executed if, and only if, command1 finishes successfully (returns 0 exit status)
command1 && command2

The OR-list has the form:

# command2 will be executed if, and only if, command finishes unsuccessfully (returns code of error)
command1 || command2

The return code of AND and OR lists is the exit status of the last executed command.

Conditional statements

Like in other languages, Bash conditionals let us decide to perform an action or not, depend on result by evaluating an expression, which should be enclosed in [[ ]].

Conditional expression may contain && and || operator, which are AND and OR accordingly. Beside this, there many other handy expression.

There are two different conditional statements: if statement and case statement.

Primary and combining expressions

Expressions enclosed inside [[ ]] are called test commands or primaries. These expressions help us to indicate result of an conditional.

Working with file system:

Primary	Meaning
`[ -e FILE ]`	True if `FILE` exists and is a directory or regular file.
`[ -f FILE ]`	True if `FILE` exists and is a regular file.
`[ -d FILE ]`	True if `FILE` exists and is a directory.
`[ -s FILE ]`	True if `FILE` exists and not empty (size more than 0).
`[ -r FILE ]`	True if `FILE` exists and is readable.
`[ -w FILE ]`	True if `FILE` exists and is writable.
`[ -x FILE ]`	True if `FILE` exists and is executable.
`[ -L FILE ]`	True if `FILE` exists and is symbolic link.
`[ FILE1 -nt FILE2 ]`	FILE1 is newer than FILE2.
`[ FILE1 -ot FILE2 ]`	FILE1 is older than FILE2.

Working with strings:

Primary	Meaning
`[ -z STR ]`	`STR` is empty (the length is zero).
`[ -n STR ]`	`STR` is not empty (the length is non-zero).
`[ STR1 == STR2 ]`	`STR1` and `STR2` are equal.
`[ STR1 != STR2 ]`	`STR1` and `STR2` are not equal.

Arithmetic binary operators:

Primary	Meaning
`[ ARG1 -eq ARG2 ]`	`ARG1` is equal to `ARG2`.
`[ ARG1 -ne ARG2 ]`	`ARG1` is not equal to `ARG2`.
`[ ARG1 -lt ARG2 ]`	`ARG1` is less than `ARG2`.
`[ ARG1 -le ARG2 ]`	`ARG1` is less than or equal to `ARG2`.
`[ ARG1 -gt ARG2 ]`	`ARG1` is greater than `ARG2`.
`[ ARG1 -ge ARG2 ]`	`ARG1` is greater or equal to `ARG2`.

Conditions may be combined using these combining expressions:

Operation	Effect
`[ ! EXPR ]`	True if `EXPR` is false.
`[ (EXPR) ]`	Returns the value of `EXPR`.
`[ EXPR1 -a EXPR2 ]`	Logical AND. True if `EXPR1` and `EXPR2` are true.
`[ EXPR1 -o EXPR2 ]`	Logical OR. True if `EXPR1` or `EXPR2` are true.

Sure, there is more useful primaries and you can easily find it in Bash man pages.

Using of `if` statement

if statement works completely at the same way as it works in other programming languages. If the expression within braces is true, the code between then and fi, which indicates the end of the conditionally executed code.

# Single-line
if [[ 1 -eq 1 ]]; then echo "true"; fi;

# Multi-line
if [[ 1 -eq 1 ]]; then
  echo "true";
fi;

At the same time, we could use if..else statement such as:

# Single-line
if [[ 2 -ne 1 ]]; then echo "true"; else echo "false"; fi;

# Multi-line
if [[ 2 -ne 1 ]]; then
  echo "true";
else
  echo "false";
fi;

Sometimes if..else statements is not enough to do what we wanna do. In this case we shouldn't forget about existence of if..elif..else statement, which may be very handy. Look at the example below:

if [[ `uname` == "Adam" ]]; then
  echo "Do not eat an apple!";
elif [[ `uname` == "Eva" ]]; then
  echo "Do not take an apple!";
else
  echo "Apples are delicious!";
fi;

Using of `case` statement

If you are confronted with a couple of different possible actions to take, then using of case statement may more useful than nested if statement. For more complex conditions use case like below:

case "$extension" in
  "jpg"|"jpeg")
    echo "It's image with jpeg extension."
  ;;
  "png")
    echo "It's image with png extension."
  ;;
  "gif")
    echo "Oh, it's a giphy!"
  ;;
  *)
    echo "Woops! It's not image!"
  ;;
esac;

Each case is an expression matching a pattern. The | sign is used for separating multiple patterns, and the ) operator terminates a pattern list. The commands for the first match are executed. * it's pattern for anything else, than doesn't match with defined patterns. Each block of command should be divided with ;; operator.

Loops

Here we won't be surprised. As in any programming language, a loop in bash is a block of code that iterates as long as the control conditional is true.

There are three types of loops in Bash: for, while, until and select.

`for` loop

The for is very similar to its sibling in C. It looks like this:

for arg in elem1 elem2 ... elemN
do
  # statements
done

During each pass through the loop, arg takes on the value from elem1 to elemN. Instead of these value may be wildcards or brace expansion.

Also, we can write for loop in one line, but in this case there needs to be semicolon before do, like below:

for i in {1..5}; do echo $i; done

By the way, if for..in..do seems a little bit weird for you, as well you can write for in C-like style such as:

for (( i = 0; i < 10; i++ )); do
  echo $i
done

for is handy when we wanna do the same operation over each file in directory. For example, if we need to move all .bash files into script folder and then give them execute permissions, our script would look like this:

#!/bin/bash

for FILE in $HOME/*.bash; do
  mv $FILE ${HOME}/scripts
  chmod +x ${HOME}/scripts/${FILE}
done

`while` loop

The while loop tests a condition and looping till that condition is true. Condition is nothing more than primary is used in if..then conditions. So while loop looks like this:

while [[ condition ]]
do
  # statements
done

As in the same case with for, if we want to write do and condition in the same line, then should be a semicolon before do.

Working example might be look such as:

#!/bin/bash

# Squares of numbers from 1 to 10
x=0
while [[ $x -lt 10 ]]; do # value of x is less than 10
  echo $(($x*$x))
  x=`expr $x + 1` # increase x
done

`until` loop

The until loop is opposite to while loop. Like a while it check test condition, but it keeps looping as long as this condition is false:

until [[ conditions ]]; do
  #statements
done

`select` loop

The select loop help us to organize a user menu. It has almost the same syntax as for loop:

select answer in elem1 elem2 ... elemN
do
  # statements
done

The select print all elem1..elemN on the screen with their sequence numbers, after that it prompts the user. Usually it looks like #? (PS3 variable). The answer will save in answer. If answer is the number between 1..N, then statements will execute and select will go to the next iteration — that's because we should use break statement.

The working example might look like this:

#!/bin/bash

PS3="Choose the package manager: "
select ITEM in bower npm gem pip
do
  echo -n "Enter the package name: " && read PACKAGE
  case $ITEM in
    bower) bower install $PACKAGE ;;
    npm)   npm   install $PACKAGE ;;
    gem)   gem   install $PACKAGE ;;
    pip)   pip   install $PACKAGE ;;
  esac;
  break # avoid infinite loop
done

This example, as user what package manager would he/she like to use. After that it will ask what package we want to install and finally install it.

If we run, we will have:

$ ./my_script
1) bower
2) npm
3) gem
4) pip
Choose the package manager: 2
Enter the package name: bash-handbook
<installing of bash-handbook>

Loop control

There are situation when we need to stop loop before its normal ending or step over iteration For these cases there are built-in break and continue statements and both of them work with every kind of loops as well.

The break statement is used to exit the current loop before its ending. We have already met with it.

The continue statement steps over one iteration. We can use it such as:

for (( i = 0; i < 10; i++ )); do
  if [[ $(($i % 2)) == 0 ]]; then continue; fi;
  echo $i
done

If we run example above, it will print all odd numbers from 0 to 10.

Functions

In scripts we have ability to define and call functions. As in any programming language, functions in bash it's a pieces of code, but there are differences.

In bash functions it's a named group of commands. Calling a function is just like calling another program, you just write its name.

We can declare our own function looks like below:

my_func () {
  # statements
}

my_func # call my_func

Declaring of a function should be before the first call.

Functions can take on arguments and return result — exit code. Arguments, within functions, are treated in the same manner as arguments given to the script in non-interactive mode — using positional parameters. Result can be returned using return command.

So below is an function that takes name and return code of success:

# function with params
greeting () {
  if [[ -n $1 ]]; then
    echo "Hello, $1!"
  else
    echo "Hello, unknown!"
  fi
  return 0
}

greeting Denys  # Hello, Denys!
greeting        # Hello, unknown!

We've already mentioned about exit codes. The return command returns the exit code of the last executed command. Above, return 0 will return successful exit code.

Debugging

The shell give us tools for debugging scripts. If we want to run script in debug mode, script's shebang should have a special option:

#!/bin/bash options

These options are settings that change shell behavior. The following table is a list of options which might be useful for you:

Short	Name	Description
`-f`	noglob	Disable filename expansion (globbing).
`-i`	interactive	Script runs in interactive mode.
`-n`	noexec	Read command, but don't execute them (syntax check).
`-t`	—	Exit after first command.
`-v`	verbose	Print each command to `stdout` before executing it.
`-x`	xtrace	Print each command to `stdout` before executing it and expands commands.

For example, we have script with -x option such as:

#!/bin/bash -x

for (( i = 0; i < 3; i++ )); do
  echo $i
done

It will print to stdout value of variables and other useful information:

$ ./my_script
+ (( i = 0 ))
+ (( i < 3 ))
+ echo 0
0
+ (( i++  ))
+ (( i < 3 ))
+ echo 1
1
+ (( i++  ))
+ (( i < 3 ))
+ echo 2
2
+ (( i++  ))
+ (( i < 3 ))

Sometimes we need to debug part of script. Here the set command comes to help us. This command can enable and disable options. Options are turned on using - and turned off using +:

#!/bin/bash

echo "xtrace is turned off"
set -x
echo "xtrace is enabled"
set +x
echo "xtrace is turned off again"

Afterwords

I hope this small handbook is really interesting for you. In general, I wrote this handbook for myself to not forget the basic of Bash. I tried to write concisely but meaningfully, so I believe you appreciate it.

This handbook consists of my own experience with Bash. It does not purport to be comprehensive, so if you want more, please use man bash.

I am trying be open to any thoughts. I will be very grateful for any adjustments, corrections or question. Please, send them to Issues.

Thanks for reading!