Be Lazy: Automate Everything

06 Aug 2008 Be Lazy: Automate Everything

One of my pet hates in software development is repetitive tasks: a complicated deployment process, tricky configuration of an application, repetitive editing motions that are just a little too messy for a find/replace. All of the above have, in the past, killed my concentration, dulled my senses and otherwise numbed my brain. I quickly realized that learning the ins and outs of a few powerful tools can take away a lot of the drudgery that sometimes crops up in my day-to-day work. Further, combining these tools makes it possible to automate just about everything that I need to do on a regular (or even not-so-regular) basis.

What’s important about everything that follows isn’t necessarily the process/tools itself, but what you get out of learning them. This is just what works for me, what has accumulated over a few years.

0. Pick a Good Editor (or IDE)

This first point is a half-hearted gesture because I don’t actually expect anyone to listen to me.

Use vim.

In all seriousness, if your editor doesn’t support basic macros and/or keyboard recording you’re going to encounter situations where you get slowed down by dull, repetitive editing. Even in an IDE that supports refactoring, macros will save you time and brain-dead copy/pastes. Automating repetitive tasks in editing source code is one of the simplest ways you can help yourself.

1. Learn Your Shell (or Learn Your IDE)

I think the Pragmatic Programmer said it well before I even knew what a “shell” was: the command-line is powerful.

Becoming very familiar with a powerful shell is one of the best things you can do for yourself as a developer. Yes, the syntax can be funky and the semantics confusing but over time you come to love it. For me, it’s the Unix shell: in *nix, the shell becomes the glue between all the other tools you use to get work done — including custom scripts and programs that take care of the heavy lifting for you. Windows users might be interested in PowerShell, which is a similar offering from Microsoft — you have the power of the .NET framework at your fingertips.

I understand, too, that folks accustomed to IDEs tend to get a little irked by shell programming. I still strongly recommend giving it a go, but scripting or extending your IDE to automate certain tasks you perform on a regular basis is a great start. If you find you lose a lot of time uploading and modifying configuration files to various servers, you may want to write an Eclipse plugin to make that process as simple as the click of a button.

Whatever path you choose — learning the more exotic aspects of a shell or the plugin API of your favourite IDE — learn it well. The only way to learn it well is to use it all the time.

2. Learn a Dynamically Typed Programming Language

If your shell scripts start to look like a train wreck when you need to perform complex operations, you may want to consider falling back to a scripting language: Perl, Python and/or Ruby are all great choices — bonus points if you mix the three to meet your needs. Languages like these will give you a more structured environment in which to express what you’re trying to achieve (at the cost of it being slightly more difficult to invoke external processes than, say, Bash — a standard Unix shell).

I think it’s important to note that I mention “dynamically typed” languages here because they generally don’t have a separate compilation step: bytecode compilation occurs as part of the execution of a Perl/Python/Ruby program. Thus, a small change doesn’t require an ant/make call to pull your code together again. Remember: you’re trying to automate a process, not create more process. Build management is overhead.

By the same token, if your program needs low-level access to hardware or raw speed for processing large quantities of data — don’t be afraid to fall back to a language like C or Java if you’re sure of your reasons for doing so. Needless to say, situations such as this are rare but not impossible.

3. Appreciation of the Supporting Cast

ssh, scp, rsync, svn/cvs/git, diff, patch, wget, sed, grep, find … the list of handy *nix command-line tools goes on and on. These are smaller tools that — with a good shell — can easily be combined to automate more complicated process. The output of these programs can also be piped into a script of your own doing to perform more complicated operations; then, the output of that program can be piped into another and so on.

This is the big thing I miss when working in Windows: even with PowerShell in play to alleviate the pain that is DOS, it has terrible tool support out of the box. Having the .NET framework on hand is great, but even that’s no substitute for the *nix command-line tools mentioned above. Rant aside, there are ways around most of the shortcomings in Windows and it’s possible to emulate some of the more useful command-line *nix tools. It just means you may need to put in more effort than a *nix counterpart at times.

For folks interested in automating with their IDE, your “supporting cast” might mean third party Java/.NET libraries which your automation plugin(s)/scripts can use to perform more complicated operations.

4. Invest Time in Learning Your Tools

Really, it all comes down to learning the tools from steps 1-3 backwards. Windows, Linux or OSX — you should be an expert in how your tools work and ideally use them on a daily basis. Really getting to know your tools might mean stepping out of your comfort zone a little to learn the nuts and bolts of how your shell ticks, or taking some time out of your day to learn certain esoteric API calls, or the semantics of some new construct in your scripting language of choice.

The goal of such practice is to surround yourself with an easily extensible suite of tools where every action feels like it’s an obvious choice. These tools will help to take away some of the unavoidable “noise” on the fringes of any software development project, and let you get on with getting your job done.

5. Write Lots of Scripts to Automate Common Tasks

This is at the core of automation: combine the above tools to get things done. Copy files to a server using scp, have your script extract and install it via ssh, manipulate configuration files using sed, grep and perl before finally starting the application and storing the process ID in a file with a quick shell command. All without you typing much more than “deploy”.

Automated tasks should be easy to kick off, require little-to-no input from you and — ideally — be fast, so you don’t have to wait long to determine if anything went haywire.

Automate at the First Sign of Resistance

Once you become accustomed to your tools — and again, by no means am I implying that those described here are the best or the only — you’ll step back when you start feel some resistance performing a task throughout your day:

Hmm. I don’t want to have to go through that 12-step deployment process all over again.

It becomes second nature to whip up a script that not only does the complicated deployment but verifies that the application is up and running as expected, checking a process exists and scanning the logs for hints indicating that everything’s fine. Over time you should find your automated tasks are easier to write and “feel” much more reliable than any manual process.

And, hopefully, you’ll be happier for it. 🙂