Bash scripting without hating your life

I’ve been doing some bash automation recently (as I happen to do, like, once a year). And it can be completely infuriating picking up bash again after spending a long time with a real language.

Rule 0

Avoid writing bash scripts if you can.

Ok, some guy named Steve built your deploy pipeline in bash five years ago. That is not a good reason to continue this way. Modern scripting languages are great, that’s why we use them to develop our software. It’s ok to use them in your automation pipeline. Python is available everywhere. And if your project is written in ruby or with node then it’s probably not a giant hassle to have those installed on a build server also.

But if you still have a valid reason to go with bash then

Magic line

Then allow bash to help you a bit. I now start writing all of my bash scripts with this command.

set -euxo pipefail

Let’s see what it means.

set -e

A bash script is simply a set of commands to run one after another. So when one of them fails bash will simply continue the execution. This is an endless source of grief for me as a big proponent of ‘Dead programs tell no lies’ principle.

The -e option will force the script to exit immediately after a command fails.

#!/bin/bash

# 'run-cool-command' does not exist
run-cool-command
echo "and here we are"

Without set -e

line 4: run-cool-command: command not found
and here we are

With set -e

line 4: run-cool-command: command not found

Now this is already much better. Any command returning a non-zero exit code will immediately cause an exit.

set -o pipefail

The basic idea stays the same: stop everything when something fails. But set -e is not enough when you start piping the output from one command to another.

command1 | command2

This is especially useful when command2 is a kind of a command that’s always successful (say, echo).

set -u

On the subject of making bash behave like a real language, how about checking if a variable was set?

#!/bin/bash
set -u
echo "$unset_variable"

The default behaviour would be to simply print an empty string, but with set -u we get

line 5: unset_variable: unbound variable

Seeing as unset variables are a common cause of weird failures somewhere down the line, I find it extremely useful as well.

set -x

The last one and probably the biggest time saver of all.

I once realized that as I’m writing or debugging a bash script I have to print the values of some variable or expression every second line or so. Just to make sense of it all. Well, not anymore.

The -x option means bash will print every command before it is executed. More importantly, it’s gonna evaluate all the expressions used in a command, so we’re gonna see actual runtime values of arguments and variables in our logs.

#!/bin/bash
set -x

size=XL
echo $size
echo "The shirt is $(XL)"

Is going to produce such an output.

+ size=XL
+ echo XL
XL
+ echo 'The shirt is XL'
The shirt is XL

Now I’m still removing -x from the final versions that I’m gonna run on a server and replace it with conscious logging (otherwise the logs get too large and messy). But it’s still an incredibly useful option when building the scripts.