Getting Started with Shell Programming

1. How to write a shell script?

Step 1: Use any text editor. For example: vi, nano, or vscode

• It is recommended to use editors with syntax highlighting like gedit, vscode, or vim for writing shell scripts because they display colors to differentiate between characters and keywords, making it easier to spot errors.

Step 2: After writing, grant execution permissions to the script. For example:

chmod u+x script_name

• u+x grants execution permission to the user who owns the file
• You can also use chmod 755 script_name to give read and execute permissions to everyone, but write permission only to the owner

Step 3: Execute the script. Syntax:

bash script_name
sh script_name
./script_name

The structure of a shell script is as follows:

#!/bin/bash # <- the shebang line that specifies which shell to use
command … # <- command
command …
exit 0 # <- exit with success status code

Note: The exit 0 command will be explained in detail in the Exit Status section.

2. Variables in the shell

In the Linux shell, there are two types of variables:

• System variables: created by Linux. These variables are usually written in uppercase letters (e.g., $HOME, $PATH, $USER).
• User-defined variables: created and maintained by the user.

Variable definition: variable_name=value

• Some rules for variables in the shell:

1. The name starts with a letter or underscore (_).

2. No spaces before and after the equal sign when assigning a value to a variable.

3. Variables are case-sensitive.

4. You can declare a variable with a NULL value as follows: var01= or var01=""

5. Do not use special characters like ?, *, #, @, etc. in variable names.

6. To assign the output of a command to a variable, use command substitution:

   current_date=$(date)
   # or using backticks (older style)
   current_date=`date`

3. Using variables

To access the value of a variable, use the following syntax: $variable_name or ${variable_name} (the latter is preferred for clarity)

Example:

n=10
echo $n
echo ${n}  # Preferred for clarity, especially when concatenating
name="John"
echo "Hello ${name}Doe"  # Using braces to clearly separate the variable

4. echo command

Used to display text, variable values, etc.

Syntax: echo [options] [string, variables…]

Options:

-n: do not print the trailing newline character.
-e: enable interpretation of backslash escapes in the string.
\a: alert (bell)
\b: backspace
\c: suppress trailing newline
\n: newline
\r: carriage return
\t: tab
\\: backslash

Example:

echo -e "one two three \a\t\t four \n"

5. Arithmetic in Shell

expr 1 + 3
expr 2 - 1
expr 10 / 2
expr 20 % 3
expr 10 \* 3  # Note the backslash before *
echo `expr 6 + 3`
x=4
z=`expr $x + 3`

let "z=$z+3"
let "z += 3"
let "z=$m*$n"

z=$((z+3))
z=$(($m*$n))
# You can also do more complex arithmetic
result=$((10 + 5 * 2))  # Follows order of operations

expr 20 \% 3 # 20 mod 3
expr 10 \* 3 # multiplication, use \* instead of * to distinguish from the wildcard character.

a=`expr 10 \* 3`

a=`expr 10 \* 3`
echo $a
30

6. Quotes in Shell

There are three types of quotes in shell:

": Double quotes - Variables and command substitutions are expanded, but most special characters remain special.
': Single quotes - Everything inside is taken literally, no expansions occur.
`: Backticks (or $(command)) - Execute the enclosed command and substitute its output.

Example:

name="World"
echo "Hello $name"      # Outputs: Hello World
echo 'Hello $name'      # Outputs: Hello $name
echo "Today is `date`"  # Outputs: Today is Wed Jun 12 14:23:45 PDT 2024
# Modern alternative to backticks
echo "Today is $(date)" # Outputs: Today is Wed Jun 12 14:23:45 PDT 2024

7. Exit Status

By default, in Linux, when a command or script is executed, it returns a value to determine whether the command or script was executed successfully.

1. If the return value is 0 (zero) -> the command was successful.
2. If the return value is non-zero (1-255) -> unsuccessful, with different values indicating different types of errors.

This value is called the Exit Status.

To find out the return value of a command or script, use the special shell variable: $?

Example: If you delete a non-existent file on the hard drive

rm unknown_file
echo $?  # Will print a non-zero value, typically 1

You can use exit status in conditional statements:

if command; then
    # Command succeeded (exit status was 0)
    echo "Command succeeded"
else
    # Command failed (exit status was non-zero)
    echo "Command failed with exit status $?"
fi

8. read command – read input from the keyboard, file, etc.

Used to get input from the keyboard and store it in a variable.

Syntax: read [options] var1 var2 var3 … varN

Common options:

-p "prompt": Display a prompt before reading input
-s: Silent mode (don't echo characters, useful for passwords)
-t seconds: Timeout after specified seconds
-r: Raw mode (backslashes are not treated as escape characters)

read without parameters -> the value will be stored in the $REPLY variable.

Example:

read -p "Enter your name: " name  # Displays prompt and stores input in $name
read -s -p "Enter password: " password  # Silent mode for password input
echo
read  # No variable specified, input stored in $REPLY
var="$REPLY"

Normally, the backslash \ allows line breaks to continue entering data in read. If read -r is used, it will treat backslashes literally.

Example:

read var
first line \
second line
echo "$var"
first line second line # <- result

What about with the -r parameter?

read -r var
first line \
echo "$var"
first line \  # Backslash is treated literally

The read command can be used to read from a file. If the file contains more than one line, only the first line will be assigned to the variable.

If read has more than one variable (read var1 var2 …), read will rely on the $IFS (Internal Field Separator) variable to assign data to the variables.

By default, $IFS consists of space, tab, and newline characters.

Example:

read var < data_file  # Reads first line of data_file into var

If the file has more than one line:

read var1 var2 < data_file

In this case, each variable will contain a string separated by the $IFS value, not an entire line. The last variable will contain the rest of the line.

To read the entire file, use a loop:

while read line
do
  echo "$line"
done < data_file

To customize how read splits input, modify the $IFS variable:

echo "list all users"
OIFS=$IFS  # Backup original IFS
IFS=:      # Set IFS to colon for /etc/passwd file format
while read name passwd uid gid fullname ignore
do
  echo "$name $fullname"
done < /etc/passwd
IFS=$OIFS  # Restore the original IFS

A cleaner approach is to set IFS only for the scope of the read command:

while IFS=: read name passwd uid gid fullname ignore
do
  echo "$name $fullname"
done < /etc/passwd
# IFS remains unchanged outside the loop

9. Command-line Arguments

When you run a script with arguments, you can access those arguments using special variables:

$0  # The name of the script itself
$1  # The first argument
$2  # The second argument
$3  # The third argument, and so on...
$#  # The number of arguments passed
$@  # All arguments as separate strings: "$1" "$2" "$3" ...
$*  # All arguments as a single string: "$1 $2 $3 ..."

Example: Let’s say we have a script named myself, to execute this script with arguments:

./myself one two

In the script, you can access these parameters as follows:

echo "Script name: $0"          # Outputs: Script name: ./myself
echo "First argument: $1"       # Outputs: First argument: one
echo "Second argument: $2"      # Outputs: Second argument: two
echo "Number of arguments: $#"  # Outputs: Number of arguments: 2
echo "All arguments: $@"        # Outputs: All arguments: one two

You can process all arguments in a loop:

for arg in "$@"
do
  echo "Argument: $arg"
done

10. Redirection

Most commands output results to the screen (standard output) or take input from the keyboard (standard input), but with Linux, you can redirect these streams.

There are three standard streams:

• Standard input (stdin): File descriptor 0
• Standard output (stdout): File descriptor 1
• Standard error (stderr): File descriptor 2

Common redirection operators:

1. > - Redirect stdout to a file (overwrite)
2. >> - Redirect stdout to a file (append)
3. < - Redirect stdin from a file
4. 2> - Redirect stderr to a file
5. &> - Redirect both stdout and stderr to a file

Examples:

ls > filename                # Redirect stdout to filename (overwrite)
ls >> filename               # Append stdout to filename
cat < input.txt              # Read from input.txt instead of keyboard
command 2> error.log         # Redirect stderr to error.log
command > output.txt 2>&1    # Redirect both stdout and stderr to output.txt
command &> all.log           # Shorter syntax for redirecting both stdout and stderr

You can also redirect to /dev/null to discard output:

command > /dev/null          # Discard stdout
command 2> /dev/null         # Discard stderr
command &> /dev/null         # Discard both stdout and stderr

11. Pipes

Pipes allow you to use the output of one command as the input to another command.

Syntax: command1 | command2

Examples:

ls -l | grep ".txt"          # List files and filter for .txt files
cat file.txt | sort          # Sort the contents of file.txt
ps aux | grep firefox        # Find firefox processes
history | grep git           # Find git commands in history
ls -l | wc -l                # Count number of files/directories

You can chain multiple pipes together:

cat /var/log/syslog | grep ERROR | sort | uniq -c

12. Conditional Statements

if [ condition ]; then
    # commands if condition is true
elif [ another_condition ]; then
    # commands if another_condition is true
else
    # commands if all conditions are false
fi

if [ -f "$filename" ]; then
    echo "File exists"
elif [ -d "$filename" ]; then
    echo "It's a directory"
else
    echo "File does not exist"
fi

case $variable in
    pattern1)
        # commands for pattern1
        ;;
    pattern2)
        # commands for pattern2
        ;;
    *)
        # default commands
        ;;
esac

case $answer in
    [yY]|[yY][eE][sS])
        echo "You answered yes"
        ;;
    [nN]|[nN][oO])
        echo "You answered no"
        ;;
    *)
        echo "Invalid answer"
        ;;
esac

13. Loops

for variable in list
do
    # commands
done

# Loop through a list of values
for name in John Mary Steve
do
    echo "Hello $name"
done

# Loop through files
for file in *.txt
do
    echo "Processing $file"
done

# Loop through numbers
for i in {1..5}
do
    echo "Number: $i"
done

# C-style for loop
for ((i=1; i<=5; i++))
do
    echo "Count: $i"
done

while [ condition ]
do
    # commands
done

count=1
while [ $count -le 5 ]
do
    echo "Count: $count"
    count=$((count+1))
done

until [ condition ]
do
    # commands
done

count=1
until [ $count -gt 5 ]
do
    echo "Count: $count"
    count=$((count+1))
done

Conclusion

These are some basic shell programming concepts that should help you get started with writing your own shell scripts. Shell scripting is a powerful tool for automating tasks in Linux and Unix-like operating systems.

Some additional resources to continue learning:

• Bash Guide for Beginners
• Advanced Bash-Scripting Guide
• ShellCheck - A tool to find bugs in your shell scripts

Happy scripting!

Translated from Vietnamese version by ChatGPT