Two years ago, I started as a contractor at SGI, herding unix machines and with the general guideline of “do cool things and train the rest of the team on sendmail, when you have time”. I was hired into a group of five sysadmins, one “monitoring” guy, two “remote access” folks, one security guy and a manager.

Since that time I have:
deployed a spam appliance
trained people on spam
deployed a dhcp management interface and 4 dhcp servers
trained people on dhcp
taken over “monitoring” (read: nagios and munin) from the “monitoring guy” when he left
trained people on monitoring
seen two sysadmins get laid off
got converted from contract to perm right near a layoff, hard to do
taken over design/architecture roles when 4 designers left within a couple months
seen our manager leave and not get replaced
taken over “remote access” when the last remote access person left (a @!*&load of busy-work)

Basically, I’ve done everything I can think of to do except for training people on sendmail :)

Now, today, two more sysadmins were laid off. That means of our original group of 10 we are left with one sysadmin, and one security guy. (Thank goodness he is a good sysadmin too, I just have to show him where the tools are). I am, by default, the shaman: the inheritor of a rich, 15+ year oral history of how we got to this point, and I’ve only been here 2 years. Winning at this version of musical chairs is not fun.

This looks like the beginning of the end, for me. I still have a couple things I want to accomplish in the next 3-4 months, so that I can say I did them, but if I continue to get piled with busywork with 0 need for creativity and 0 opportunity to change things for the better, I’ll probably leave close to the end of the year whether or not I accomplished the things I wanted to do.

The next big thing? I’m not sure, but hopefully it will involve killing spammers and perl coding.

Those who were interested in my “pusher” script, here is a “sneak preview”. Pusher Pre-Alpha Preview

Right now it knows how to install ~root/authorized_keys, push a firewall script and run it, and push apache2 config files.

It is very debian-centric and will need some tweaking to allow alternate handling of /etc/init.d scripts for other platforms, as well as taking some templates and variable definitions and putting them in a central “global template” location.

Additional things to do include: a package installer to apt-get things (checking if they are already installed), a “file mover” to sync various directories from one host to another, and a munin-node config package.

Enjoy? Comments welcome. Smart people should please mock my design (constructively of course :)

One thing I still miss about AV is “fetch”. Fetch was not just a program; it was a freakin’ way of life. All machines ran fetch. All machines obeyed the great and powerful fetch.

Fetch was a perl script made for configuring a lot of machines identically. It could literally install, configure, and start up all the services needed on a new box. Pretty much all we had to do was install the server using the OS CD (DEC or Linux), teach the new machine its hostname, and download and run this program.

Since leaving AV I haven’t really found anything like it. Now I find myself yearning yet again for “fire and forget” machine management. If I am going to get serious about my own private hosting setup, I’m going to need a mechanism for installing machines from scratch, quickly, and predictably, without a lot of manual steps.

Nature abhors a vacuum, and so do I. So instead of vacuuming, I found myself working on a new world order.

I have dubbed the new world order “pusher”. gory details for geeks

Posted for review, comments and entertainment value. If this is interesting, please comment, or just point and laugh; it’s okay.

Casual Sysadmin Hosting Service

“Don’t ignore your domain. We’ll do it for you!”

Service overview

We host a handful of vanity domains for ourselves and friends. We try to do everything as cheaply as possible and we pass the savings on to you.
Continue reading →

I find that I am growing weary of maintaining my own servers at home, but I’m still way too independent to turn control of my domains over to a “web hosting” service. So, I am starting to look at some of the “virtual server” hosting options. These are services that you pay something like $20-40 a month and you get a “virtual” server (it runs linux, so you can run mysql, mud/mush, procmail, spamassassin, apache, php, gallery, and all the fun stuff you would run at home without having a space-heater in your spare room 24×7. It’s not a “real” server in the sense of dedicated hardware, it’s really a virtual machine running on a larger, beefier machine and capped to a small slice of cpu and memory.

So, question one, does anyone have experience with these type of “virtual server” hosting arrangements? I’d be interested in hearing your thoughts on the experience of shopping for one, using one, and whether you have had to call for support and how the service was.

Also, for the folks who have domains hosted on a friend’s server (mine or some other friend) would you consider paying your friend something like $20 a year to maintain the web/mail/whatever for your domain? This would be in addition to the $8-10 or whatever you pay for the domain itself. (By comparison, basic “domain hosting” which includes certain amount of disk space, certain amount of bandwidth, bunch of email accounts, probably runs in the $4-6/month range, so that would be $50-70 per year. These often include php/cgi and some mysql databases, but would not include shell access/tf/screen, procmail, mailing list hosting, mud/mush or other one-offs.)

I’m thinking it would be great if I could get 5-8 or so friends to contribute $20 per year, possibly a bit more for multiple domains, mailing lists, databases, forums or other “power” features, that would probably be ideal. At first I would probably keep the servers I have here at home running in tandem (and as a backup) but eventually the goal would be to have everything virtual, or if we find enough friends, maybe two different virtual servers at different sites…

Thoughts?

We need to find homes for some items. Please let me know if you’re interested in any of these. (email addr on my profile works best).
Futon bed frame, stereo components, computer desk, tv stand, exercise bike, cd player jukebox