I just learned a really good bash trick which is something I've wanted to have but didn't really appreciate that it was possible so I never even searched for it.
set -ex
Ok, one thing at a time.
set -e
What this does, at the top of your bash script is that it exits as soon as any line in the bash script fails.
Suppose you have a script like this:
git pull origin master find . | grep '\.pyc$' | xargs rm ./restart_server.sh
If the first line fails you don't want the second line to execute and you don't want the third line to execute either. The naive solution is to "and" them:
git pull origin master && find . | grep '\.pyc$' | xargs rm && ./restart_server.sh
but now it's just getting silly. (and is it even working?)
What set -e does is that it exits if any of the lines fail.
set -x
What this does is that it prints each command that is going to be executed with a little plus.
The output can look something like this:
+ rm -f pg_all.sql pg_all.sql.gz + pg_dumpall + apack pg_all.sql.gz pg_all.sql ++ date +%A + s3cmd put --reduced-redundancy pg_all.sql.gz s3://db-backups-peterbe/Sunday/ pg_all.sql.gz -> s3://db-backups-peterbe/Sunday/pg_all.sql.gz [part 1 of 2, 15MB] 15728640 of 15728640 100% in 0s 21.22 MB/s done pg_all.sql.gz -> s3://db-backups-peterbe/Sunday/pg_all.sql.gz [part 2 of 2, 14MB] 14729510 of 14729510 100% in 0s 21.50 MB/s done + rm pg_all.sql pg_all.sql.gz
...when the script looks like this:
#!/bin/bash set -ex rm -f pg_all.sql pg_all.sql.gz pg_dumpall > pg_all.sql apack pg_all.sql.gz pg_all.sql s3cmd put --reduced-redundancy pg_all.sql.gz s3://db-backups-peterbe/`date +%A`/ rm pg_all.sql pg_all.sql.gz
And to combine these two gems you simply put set -ex at the top of your bash script.
Thanks @bramwelt for showing me this one.
UPDATE
Checkout out ExplainShell.com
Comments
Post your own commentuseful for small scripts and debugging, but the proper way to ensure that a command only executes if the previous command succeeds is to use the && you seem to not like.
using your methodology, my script would simply die if any exit code were non-zero, but one should not rely on this. so, say I'm setting a variable to the output of a command. this would result in a return code of zero if the assignment succeeded, not just if the command's exit code were zero.
i recommend you investigate further into some bash best practices. set -e is good for debugging, but once a script is working as expected, you'll want to remove set -e in most cases and catch failures within the script to take appropriate action, not just always bail out.
The commentary on set -e behaviour is a somewhat lacking. To begin with, the concept of a failing line is nebulous. Lines don't fail but commands contained with them can. This distinction is important.
Regarding the sample script, the author implies that the goal is to exit if the git command fails, in which case the appropriate solution is this:
git pull origin master || exit
Now, if git returns a non-zero exit status, the script will exit with the very same status. It's far more obvious what will happen from reading the script and there is no having to contend with the numerous side effects of enabling set -e (more on that below).
Next, the use of the && operator is mentioned as a possible solution:
git pull origin master && find . | grep '\.pyc$' | xargs rm && ./restart_server.sh
This approach has nothing in common with set -e because a failing command will not cause the script to exit. Further, the exit status of both find and grep will be disregarded, which is probably not what the author is expecting. In Bash, this deficiency can be addressed with set -o pipefail, in which case the pipeline returns the exit status of the rightmost command that failed, or zero if *all* commands succeeded. Still, in this case, a better approach would be to avoid using pipes at all:
git pull origin master && find . -name '*.pyc' -delete && ./restart_server.sh
For that matter, using find -delete is safer because it will cope with filenames that contain a newline character (as unlikely as that may be).
In summary, I recommend explicitly using the exit builtin if the failure of a given command should cause the script to exit. Not only does set -e result in action at a distance, but it also brings with it a surprising number of pitfalls that are only understood by experienced Bash practitioners. For more information on this topic, refer to http://mywiki.wooledge.org/BashFAQ/105.
Thanks for your insightful response. Much appreciated.
Yeah, explicitness is a good thing and peppering your script with `...|| exit` gives it a chance to make some things exit on failure but allow others to fail and that's fine.
The other nice option to set is `-u` which forces a script to quit if it will find an undefined variable.
Thanks! That's really useful.
it seems great until you find something what doesn't work logically. Here's my problem https://stackoverflow.com/questions/61675304/bash-set-e-why-it-behaves-couterlogic
Because https://stackoverflow.com/a/40532535/205832
thanks for the post. found it useful in 2020.