Why Bash for MyCmd?

I have been asked, “Why Bash?” when describing MyCmd and its implementation.

To be frank, much of it is inertia – I’ve been writing shell scripts for over 25 years. I have attempted to write my own “standard library” for Bash several times over the years, MyCmd is just my latest (and hopefully, last) incarnation of this idea.

MyCmd’s Use Cases

It is important to understand the use cases for a piece of software to be able to make the right technology choices. For me, MyCmd is a tool to glue together other command line tools. Here are a few of the things I’ve written with it (and will rewrite with my latest version, when complete):

• A tmux session manager with a plugin system that was aware of the custom build systems used at my former job
• Calls to REST-like APIs using curl to gather quick data
• A project task runner; tasks are defined as shell functions
• Scripts to import financial data into my accounting system
• A tool to execute builds locally and capture the output in logs
• My daily startup routine – sign my ssh keys with 2FA, log into VPN, pull my day’s calendar into Emacs Org-mode, etc.

As you can see, most of my use cases for MyCmd commands are wrappers around command line tools. Because of this, I want to optimize for easily executing external programs.

For example, see this function for updating my SSH keychain:

set -o nounset -o errexit -o errtrace -o pipefail

function daily.update_ssh_keychain() {
    # First, remove any expired SSH credentials
    ssh-add -D

    # Then re-add work and personal credentials, if present
    local identity

    for identity in id_ecdsa personal_id_rsa; do
        if [[ -e "${HOME}/.ssh/${identity}" ]]; then
            ssh-add "${HOME}/.ssh/${identity}"
        fi
    done
}

A similar function in Python, that replicates the above behavior:

import subprocess
from pathlib import Path

def update_ssh_keychain():
    subprocess.run(["ssh-add", "-D"], check=True)

    for identity in ["id_ecdsa", "personal_id_rsa"]:
        keyfile = Path(f"~/.ssh/{identity}").expanduser()

        if keyfile.exists():
            subprocess.run(["ssh-add", keyfile.as_posix()], check=True)

Although this simple example may not seem like it adds a lot of friction, but with a lot of calls to external programs, the calls to subprocess.run end up being laborious.

Additionally, many of the commands that I implement with MyCmd start out of me just fiddling around in an interactive shell. It becomes easy, therefore, to just take that code and turn it into something more robust and reusable with MyCmd.

What about POSIX and other shells?

If I insist on writing shell scripts, why not target POSIX shell or Zsh instead of Bash? Because, in fact, I actually have been using Zsh everywhere for my interactive shell for a few years. Why not use these? It comes down to a few things:

• Modern Bash has a lot of features that I use that are not available in POSIX, including, but not limited to the following:
  • Arrays
  • Associative Arrays
  • Array References
• My use case is to write tools to be used interactively, which for me means a modern Linux or Mac OS system, where Bash is easily available or installable.
• A lot of the tools that I use to aid in my development, such as shfmt from mvdan/sh do not support the Zsh dialect.
• Once again, to be frank, it’s what I’m used to.

I do want to also call out that MyCmd is not pure shell. I do rely on a few external tools, including those from GNU coreutils. I have built-in support in MyCmd to handle finding and executing tools in a cross platform way.

Alternatives

I am not tied to Bash. If I find a language or tool that I think fits my goals, I would happily rewrite MyCmd in that.

• Janet: The Janet Language looks to be an interesting Lisp-derived language. The janet-sh library does look like it makes calling external processes relatively easy. I intend to explore the language a bit more. I am a long-time Lisp user, so this is an attractive idea. However, I believe making Janet a prerequisite might be a bit bigger barrier for my future end-users and MyCmd Command and Command Group writers than Bash. Janet is not available in a lot of package managers yet and so installing it will require a compile. I am still open to this as an idea.

• cmdio, op, and ops: My friend Chris Lesiw has written some pretty amazing libraries in Go to help automate his build processes, including cmdio, op, and ops. The cmdio documentation has some examples of how you can construct pipelines and execute external programs. These are nice APIs and if I were only writing automation for building Go projects they would be a strong contender. However, MyCmd is more general and I really don’t want to take a dependency on Go and a compilation step for MyCmd commands and command groups.

• ysh from Oils: The Oils project is one that I have been keeping an eye on for a while. It is implementing two languages, osh as an easy upgrade from Bash and POSIX shells, and ysh as a more modern shell language. There are many features of ysh that are appealing to me. It still feels early on in the development of ysh and would probably require compiling it to deploy to various places I might want to run MyCmd. However, I am keeping my eyes on this for the future. This is probably the one that is most interesting to me.

• NuShell: The NuShell project is a new shell written in Rust that makes manipulating and using data a lot easier. It seems pretty focused on the interactive experience, though I see it does support modules and other things that would make implementing MyCmd easier. I admit I haven’t looked at it close enough to make an educated decision on why I wouldn’t use NuShell.

I have explored a few languages that transpile to Bash, such as Amber, but none of them have felt mature enough or given me enough advantage over plain Bash to warrant their use.

I have also become interested in rewriting languages like Nova and exploring if they could be used to write CLI tools either directly or by transpiling to Bash.

Why Not Bash?

I will be the first to admit that Bash is not the friendliest language. I often have to resort to writing my own tools (like the currently on the back burner bashdoc) to get features I want. Tools like shellcheck are critical to writing error-free Bash code; though there are some bugs such as issue 1225 that I run into often because of my heavy use of array references.

There are well-documented pitfalls to using the language. A few off of the top of my head:

• Function calls happen in subprocesses.
• Functions can only return exit codes. You return values by capturing the output of the function call. This can result in some awkward code in some cases.
• You can’t have multi-dimensional arrays.
• I have to have a lot of boilerplate code in all of my scripts to ensure the environment is set up right.

Conclusion

Ultimately, I have developed a style of writing Bash that I like. It has made it easy to write the code that I want and easily create the tools and automation for my projects with it. I fully admit my choice of Bash for MyCmd is fully influenced by inertia and habit of decades of writing shell scripts. I have built useful tools with it, and I am having fun doing so, and I think that’s the most important part.