Minor Gripe

2019-10-14 -- Making your own Bash toolbelt

Chris Ertel

Introduction

I’ve been doing a bit of work lately setting up a new machine for my job, and in the process making some scripts that can ease the process for later developers. The whole thing is written in Bash. Why might you find yourself wanting such a thing?

Bash (or some other shell, probably Bash-compatible) is available nearly everywhere, so your tooling won’t need other tooling to bootstrap it
Bash is relatively easy to use for stringing together common operations
Having a common helper toolbelt with sub-commands (like you’d see with Heroku toolbelt) can save you a tremendous amount of time
Encouraging teammates to build out commands for commons tasks insulates you if the precise details for those tasks change (say, switching from one way of tagging builds to another, etc.)

Downsides to this:

This is some pretty solid yak-shaving, though if you know what you’re doing you’ll end up with more yak and less wool
Bash is not universal–some folks will run csh, dash, ksh, zsh, fish, or some other strange thing, and supporting those environments can be…exciting
Bash is super easy to get wrong without tooling

So, having said that, let’s talk about building a toolbelt, and then how to skip that.

Building a toolbelt

So, we can build a cute little toolbelt like so (adapted from here):

#! /usr/bin/env bash

set -Ceuo pipefail

do_goose() {
    echo "honk!"
}

do_echo() {
    echo $@
}

subcommand=${1:-}
case $subcommand in
	"" )
    echo "Missing subcommand."
    ;;
  "help" )
    echo "help info would go here."
    ;;
  *)
    shift # munch subcommand
    do_${subcommand} $@
    ;;
esac

And, after chmod +xing this command, you could put it through its paces and see:

crertel@attenborough:~$ ./goose.sh help
help info would go here.
crertel@attenborough:~$ ./goose.sh 
Missing subcommand.
crertel@attenborough:~$ ./goose.sh honk
./goose.sh: line 23: do_honk: command not found
crertel@attenborough:~$ ./goose.sh goose
honk!
crertel@attenborough:~$ ./goose.sh echo honk honk honk
honk honk honk

Now, a couple of observations:

We use the #! /usr/bin/env bash instead of something else because it better respects the user’s path. This is not an unalloyed good, however.
We do set -Ceuo pipefail to make sure that: overwriting files without doing >! isn’t allowed (noclobber via C), that the script exits the first time it hits an error instead of exploding (errexit via e), that using an unbound variable forces an exit and a warning (nounset via u), and finally that if something breaks in a command pipeline we don’t keep executing everything downstream of it using its error message as stdin (pipe fail via -o pipefail). I strongly suggest you use this in all your Bash scripts until you have a particular need not to.
Because we blow up on unbound variables, we pull a sneaky and use subcommand=${1:-} to make sure that if there is no first positional argument/sub-command specified, we get an empty string.

So, that was neat, but there’s a better (and more structured) way.

Sub: a better framework for bash toolbelts

I was introduced to this by my old boss. It’s a framework called sub, originally from Basecamp but forked and tweaked and updated by various folks–I’ve linked to the one Zillow uses.

Sub is nice because it gives a clean structure to build your toolbelt:

Put script files into the libexec folder named for the sub-command you want and prefixed by the toolbelt name and an underscore . If you were making a toolbelt called gripe with a sub-command “loudly”, you’d put a file at gripe/libexec/gripe_loudly, and it would be picked up by sub.
Add Usage: and Help: comments as per the readme, and you magically get usage, summary of commands, and detailed help for users.

It should take you maybe an afternoon to build your first couple of commands. Things that will probably trip you up include: refactoring common helper functions out into their own shell scripts, properly determining the path to the shell scripts so things don’t break when you run this from other directories, figuring out how to place files relative to wherever the toolbelt is installed, etc.

These are all tractable problems, but they will be a bit frustrating. Oh, also, if you want colons in your sub-commands like gripe auth:login you can do it by putting it in the filename but know that that’s super. weird.

But, if you get this working, you can do neat things like:

Have one deploy step that builds everything, versions it, tags it, blogs about it, whatever
Have a tool to manage changelogs
Have a tool to automatically setup dev environments
Have a tool to check the weather
Have a tool to post twitter updates
Have a tool to crawl a web page, or fire off a json request

All kinds of stuff, and you can just kinda load it down over time with things you find helpful. It’ll serve you well, and because it’s in naked bash, it won’t break when you put it onto a machine with weird programming environments. You don’t need Node! You don’t need Python! You don’t need Ruby!

Further bash resources

Some things that I’ve found very useful or inspiring along these lines:

The OG Heroku Toolbelt is what first got me thinking about toolbelts.
Shellcheck is a Bash static code linter and analyzer. Use this, it will save you so many bugs and so much hate. Many distros, including definitely Ubuntu and Debian, already have this in their package managers.
Shfmt is a Bash code formatter. It makes all your Bash code look the same, at least as much as it can. It’s in Go, and though I don’t think most distros have it there are a bunch of precompiled binaries, because Go.
The Advanced Bash-Scripting Guide is kind of a super-sampler of techniques for doing different types of things in Bash. Highly recommend studying.
Old Monk 7 Year Rum is just a really solid resource for any prolonged sessions doing Bash scripting. Like the command line, it’s smooth. Like your shell scripts, it’s crafted and put into a dusty corner for 7 years and ignored until somebody is in desperate need. A+, would recommend.

Tags: trade