One thing I still miss about AV is “fetch”. Fetch was not just a program; it was a freakin’ way of life. All machines ran fetch. All machines obeyed the great and powerful fetch.

Fetch was a perl script made for configuring a lot of machines identically. It could literally install, configure, and start up all the services needed on a new box. Pretty much all we had to do was install the server using the OS CD (DEC or Linux), teach the new machine its hostname, and download and run this program.

Since leaving AV I haven’t really found anything like it. Now I find myself yearning yet again for “fire and forget” machine management. If I am going to get serious about my own private hosting setup, I’m going to need a mechanism for installing machines from scratch, quickly, and predictably, without a lot of manual steps.

Nature abhors a vacuum, and so do I. So instead of vacuuming, I found myself working on a new world order.

I have dubbed the new world order “pusher”.

Pusher is a push-based system, as the name implies. Fetch used to be pull-based for various reasons that made sense at the time, I have other reasons that make sense now. Mostly I am remembering that during the gladiator era of AV, we found ourselves doing a lot of manual steps that might not have been needed if the design had been more push-based to start with.

Pusher is largely based on “make” and Makefiles. I have a fair amount of stuff I created for myself in /usr/local/adm, mostly of the type where I go and edit some config files and type “make” and the magic happens. I want to leverage that for pusher. The basic recipe for a package is going to be, rsync the package directory over to the slave, ssh to the slave, and run “make” in that directory. The basic recipe for a node will be a list of packages, and typing “make $node” will run the make for each package, with DESTHOST set to $node. That means there is a Makefile for each node, and one for each package on the pusher host, plus usually a third one that was copied to the slave host. The key to managing large numbers of Makefiles will be to include a lot of templates for common settings and operations, so the Makefile for each node and each package can be really short (instead of duplicating info from all similar Makefiles).

I want pusher to be simple yet flexible. It ought to be easy to add new nodes, add/remove packages, or update versions of packages and push to selected nodes. Fetch did this mostly by way of insane numbers of symlinks (for example “stable -> 1.0” to point to the stable version. I think I can accomplish this by placing a scant few lines in the top of the Makefile to say “If version is stable, version = 1.0” or something like that.

Here is what I got so far:

A typical node will probably have only one file, its own Makefile, listing the packages needed.

/pusher/nodes/v1.nekodojo.org/Makefile

Right now that Makefile looks like this:

DESTHOST=v1.nekodojo.org
PATH_PACKAGES=/usr/local/adm/pusher/packages

allpackages: sshkeys firewall

% : ${PATH_PACKAGES}/%/Makefile
make -C ${PATH_PACKAGES}/$@
VERSION=vnodes
DESTHOST=${DESTHOST}

Both sshkeys and firewall are made by going to the named packages/ directory and starting another make.

A typical package called “firewall” is only one file, and it’s called “firewall”. It needs to be copied to the remote machine and run.

/pusher/packages/firewall/versions/1.0/firewall
/pusher/packages/firewall/versions/vnodes/firewall
/pusher/packages/firewall/Makefile

And the Makefile for that package looks like this:

MYNAME=firewall
VERSION=default
DESTDIR=/slave/packages/${MYNAME}

actions: prep push start

prep:

push:
${DO_SSH} ${DESTHOST} mkdir -p ${DESTDIR}
${RSYNC} ${RSYNC_OPTIONS} -av versions/${VERSION}/ ${DESTHOST}:${DESTDIR}/

start:
${DO_SSH} ${DESTHOST} ${DESTDIR}/firewall

Is anyone interested in seeing this when I’ve got it ready to show off? All comments and feedback appreciated, regardless.