Packaging systems should support overlays

One of the very common things that sysadmins do when dealing with morethan one system is that we develop a common set of files that we slapon every system (or in large environments, on subsets of our systems).Sometimes this is done by hand, sometimes this is done with hand-builttools, and sometimes this is done with things like Cfengine or Puppet.

If you accept that packaging systems should be comprehensive , then packaging systems should also supportthis common need. Specifically, packaging systems should have the


Packaging systems should be comprehensive

I have a number of peculiar opinions about packaging systems in general.One of them is that if you have a good packaging system, you should beable to manage your machine entirely through it; every modification thatyou need to make should be doable as part of a package.

(Okay, not quite every modification; I'll exclude files that arein constant flux outside of your control, like /etc/passwd and /etc/shadow , and operations like rebooting the machine due tonew kernel installations.


One reason why you should not let people register other people

Let us suppose that you have a website that allows and in fact requiresregistration. Let us further suppose that you have decided to be cleverand viral; you will helpfully let people register other people, so Jimcan tell your website 'create an account for Bob, here's his emailaddress'.

(This is subtly different than registration confirmation email , at least in practice. Arguably it is worse,since you aren't even trying to confirm that Bob is interested,you're setting up the


How systems get created in the real world

From my previous entry on our DHCP portal , I mayhave left you with the impression that it sprang into existence one day,fully formed and ready to go. I regret to inform people that this wasnot at all the case; instead, our DHCP portal is the end stage of along chain of evolutionary steps.

It goes like this:

  • in the beginning, there were not even Points of Contact ; there was just a general computing supportorganization. Users contacted us directly to get their machineson the laptop

In praise of crash (and kernel source)

You might wonder how we actually found our ext2 locking issue . That is a war story too, but fortunatelyit's a relatively short one.

First, we tried to use magic SysRq commands to dumpvarious information. Let me save you some time: this doesn't work inany meaningful way. When you have several thousand processes, askingthe kernel for a magic SysRq task dump is functionally equivalent torebooting the machine, and sometimes it was literally equivalent to it.We extracted essentially no useful information


Why the Solaris packaging system is bad

I recently read this holyhandgrenade.org entry ,which rises to the defense of the Solaris packaging system on thegrounds that it's just misunderstood; on Solaris, unlike elsewhere,the packaging system is intended only for system components (andSolaris defines this narrowly), not additional software.

I disagree. The reasons I dislike the Solaris packaging system havenothing to do with how widely used it is (and how much software doesn'tcome with Solaris); I dislike it because it is, purely and simply


Patch management is part of package management

There's two approaches to distributing updates; you candistribute entire new versions of packages, or you can usesome sort of 'patches'. For essentially historical reasons , commercial Unix vendors generally use thelatter approach.

Here is something important about this: if you have patches, patchmanagement needs to be part of the packaging infrastructure, even if itis not done with the package management system and commands. It cannotand should not be something that is just slapped on over top.

Here's why:


Interesting things can happen when you scale things up

This is a sysadmin war story.

Once upon a time, there were a bunch of IMAP servers. Since this waslong ago, they were running Linux with the 2.4 kernel. They startedout storing their mail on locally attached 72 GB SCSI disks, organizedsimply with one ext2 filesystem per disk, but then they moved thestorage to a faster and more sophisticated SAN backend with RAID-10arrays (still on small fast enterprise disks), giving each server node asingle logical array (


A brief and jaundiced history of Unix packaging systems

In the beginning (in the days of V7 and BSD Unix), Unix systems cameas a great big tarball or the equivalent that included everything andyou just unpacked it onto your machine. If there were problems, peoplepassed around new versions of various bits of source in various ways;you got some, you put them on your system, you recompiled things, and soon.

Shortly after Unix vendors started selling Unix, they discoveredthat they needed some actual mechanism to deliver bugfixes andupdates to their customers


The department's model for providing computing support

In many university departments, there's a constant tension betweenworking on the computing infrastructure that is used by the entiredepartment, and working on things for individual professors and researchgroups. To put it more concretely, do you work on upgrading thedepartmental mailserver or do you set up Professor X's new cluster thatjust got dropped off at the loading dock?

(This tension is increased when Professor X's grant funding is helpingpay for computing support, as it often is. Professor X may well feel