“pusher” – a quick-and-dirty scheme to push configs to multiple machines

One thing I still miss about AV is “fetch”. Fetch was not just a program; it was a freakin’ way of life. All machines ran fetch. All machines obeyed the great and powerful fetch.

Fetch was a perl script made for configuring a lot of machines identically. It could literally install, configure, and start up all the services needed on a new box. Pretty much all we had to do was install the server using the OS CD (DEC or Linux), teach the new machine its hostname, and download and run this program.

Since leaving AV I haven’t really found anything like it. Now I find myself yearning yet again for “fire and forget” machine management. If I am going to get serious about my own private hosting setup, I’m going to need a mechanism for installing machines from scratch, quickly, and predictably, without a lot of manual steps.

Nature abhors a vacuum, and so do I. So instead of vacuuming, I found myself working on a new world order.

I have dubbed the new world order “pusher”.

Pusher is a push-based system, as the name implies. Fetch used to be pull-based for various reasons that made sense at the time, I have other reasons that make sense now. Mostly I am remembering that during the gladiator era of AV, we found ourselves doing a lot of manual steps that might not have been needed if the design had been more push-based to start with.

Pusher is largely based on “make” and Makefiles. I have a fair amount of stuff I created for myself in /usr/local/adm, mostly of the type where I go and edit some config files and type “make” and the magic happens. I want to leverage that for pusher. The basic recipe for a package is going to be, rsync the package directory over to the slave, ssh to the slave, and run “make” in that directory. The basic recipe for a node will be a list of packages, and typing “make $node” will run the make for each package, with DESTHOST set to $node. That means there is a Makefile for each node, and one for each package on the pusher host, plus usually a third one that was copied to the slave host. The key to managing large numbers of Makefiles will be to include a lot of templates for common settings and operations, so the Makefile for each node and each package can be really short (instead of duplicating info from all similar Makefiles).

I want pusher to be simple yet flexible. It ought to be easy to add new nodes, add/remove packages, or update versions of packages and push to selected nodes. Fetch did this mostly by way of insane numbers of symlinks (for example “stable -> 1.0” to point to the stable version. I think I can accomplish this by placing a scant few lines in the top of the Makefile to say “If version is stable, version = 1.0” or something like that.

Here is what I got so far:

A typical node will probably have only one file, its own Makefile, listing the packages needed.


Right now that Makefile looks like this:


allpackages: sshkeys firewall

% : ${PATH_PACKAGES}/%/Makefile
make -C ${PATH_PACKAGES}/$@

Both sshkeys and firewall are made by going to the named packages/ directory and starting another make.

A typical package called “firewall” is only one file, and it’s called “firewall”. It needs to be copied to the remote machine and run.


And the Makefile for that package looks like this:


actions: prep push start


${DO_SSH} ${DESTHOST} mkdir -p ${DESTDIR}

${DO_SSH} ${DESTHOST} ${DESTDIR}/firewall

Is anyone interested in seeing this when I’ve got it ready to show off? All comments and feedback appreciated, regardless.

7 thoughts on ““pusher” – a quick-and-dirty scheme to push configs to multiple machines

  1. nekodojo

    Truthfully, I have not checked it out at more than a passing “ask someone who knows more than I” level. What I have heard about it is that the language is a bit over-complicated, which scares me a little, but it deserves more investigation than I’ve given it.

  2. ezray

    The lack of fetch was something that Bruce and I both lamented while we were working together at Shutterfly. We do have a fairly complex cfengine system there, but the setup is nowhere near as clean as what we had with fetch. Someday I may consider setting up the host directory/symlink structure we had at AV and making cfengine play nicely with that – I think that was one of the things about our fetch setup that made it so appealing.

Leave a Reply