Improve tooling using Go

I have been using UNIX-like systems since the days of running MINIX on an 8088 machine. I’ve never been able to describe myself as a software engineer, software developer, or the like; my programming skills have always been limited to quick and dirty hacks to solve the immediate problem at hand.

I think the biggest software product I wrote was a help desk ticket tracking system, in Perl, when I suddenly and surprisingly became a help desk lead in 2000. (Fortunately my time in that role was short.)

Along those lines, I pretty much write all of my code these days in bash. (Or, more accurately, POSIX shell… I try to keep my work portable.) Mostly I’m writing quick tools for solving operations tasks or for doing one-off tests against web applications.

Beautiful Golang

Recently, I’ve discovered the beauty of Go. This is quite a stretch for me, since it’s a C-like language, and I’ve frankly done little (successful) C programming. (Most of what I’ve done was on Atmel ATMEGA microcontrollers.) I’m not a computer scientist; I can’t extoll the virtues of Go for any particular reason. It just feels sane to me.

n.b., I am still a total noob at Go.

That said, I recently decided to rewrite some of my tools that I had written in bash, in Go, just for fun. I didn’t realize there would be a practical benefit.

The old way

We don’t use DNS for our instances within AWS. We do name them, however, because they are single-purpose instances. (I already hear some grumbling about pets vs. cattle.) So, it’s helpful to be able to look them up by their Name tag. In the past, we generated .ssh/config files via a script to do this. Later, I moved to the following:

awsip() {
  # verify awscli is installed
  command -v aws >/dev/null 2>&1 || {
    echo >&2 "aws not available on PATH.";
    return 1;
  }

  host=$1
  aws ec2 describe-instances \
    --filters "Name=tag:Name,Values=${host}" \
              'Name=instance-state-name,Values=running' \
    --query 'Reservations[*].Instances[*].[PrivateIpAddress]' \
    --output text | head -1
}

awssh() {
  ip=$1
  shift
  ssh $(awsip ${ip}) $@
}

Or should I say, The slow way

awsip is pretty straightforward. Call the AWS CLI, look up the instance, output the Private IP. But, it’s horrendously slow, as the AWS CLI is Python-based:

$ time awsip foobar
172.31.23.42

real    0m3.837s
user    0m1.879s
sys     0m0.298s
$ time awsip foobar
172.31.23.42

real    0m3.584s
user    0m1.868s
sys     0m0.313s
$ time awsip foobar
172.31.23.42

real    0m3.492s
user    0m1.873s
sys     0m0.276s

3.5 seconds or so per run. That’s rather pathetic just to look up an IP address.

The Go way

awsip rewritten in Go is still pretty simple, and I’m sure a seasoned Go developer could make it even more efficient:

package main

import (
    "fmt"
    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/ec2"
    "log"
    "os"
    "path/filepath"
)

func usage() {
    fmt.Fprintf(os.Stderr, "Looks up current PrivateIpAddress of a named EC2 instance\n\n")
    fmt.Fprintf(os.Stderr, "usage: %s <instance_name>\n", filepath.Base(os.Args[0]))
}

func main() {
    if len(os.Args) != 2 {
        usage()
        os.Exit(64)
    }

    inst := os.Args[1]

    sess := session.Must(session.NewSessionWithOptions(session.Options{
        SharedConfigState: session.SharedConfigEnable,
    }))

    ec2client := ec2.New(sess)

    params := &ec2.DescribeInstancesInput{
        Filters: []*ec2.Filter{
            {
                Name:   aws.String("tag:Name"),
                Values: []*string{aws.String(inst)},
            },
            {
                Name:   aws.String("instance-state-name"),
                Values: []*string{aws.String("running")},
            },
        },
    }

    resp, err := ec2client.DescribeInstances(params)
    if err != nil {
        log.Fatal("Error listing instances", err)
    }

    if len(resp.Reservations) > 0 && len(resp.Reservations[0].Instances) > 0 {
        fmt.Println(*resp.Reservations[0].Instances[0].PrivateIpAddress)
    } else {
        fmt.Fprintf(os.Stderr, "Not found\n")
        os.Exit(1)
    }
}

The quicker way

Even for such a small program, the speed improvement is very noticible:

$ time awsip foobar
172.31.23.42

real    0m0.758s
user    0m0.073s
sys     0m0.089s
$ time awsip foobar
172.31.23.42

real    0m0.415s
user    0m0.083s
sys     0m0.093s
$ time awsip foobar
172.31.23.42

real    0m0.451s
user    0m0.078s
sys     0m0.096s

Worst case is now 0.75 seconds, a better than 75% reduction in execution time.

Next steps

As time permits, I’ll be rewriting other tools in Go that I’ve written in bash. It’s also most likely I’ll do some of the one-off webapp testing working in Go as well. httptest looks tobe a very promising starting point for this.