The importance of figuring out low-level symptoms of problems
Suppose that you have an IMAPserver; it has mirrored local system disks, a bunch of memory, and adata filesystem (where the mailboxes are) in a RAID-10 array provided bya SAN. One day, it starts falling over unpredictably; the load averagegoes to the many hundreds, IMAP service times go into the toilet,and eventually the machine has to be force-booted. But this isn'tconsistent, and when it happens it happens very rapidly, going from anormal tiny
The comedy potential inherent in people reusing your address space
For my non-sins, I used to be one of the people who saw email complaintsabout UofT-wide spam issues (as opposed to spam issues for the specificsubdomains that I was responsible for). As part of this, every so oftenwe would get a complaint saying that a UofT IP address (usually but notalways in 128.100.0.*) had spammed people, with Received: headersincluded so that we could see this ourselves.
There were two problems with
Why commands can never afford to get it wrong in a version
Netcat is a nice, handy program; there are any number of circumstancesin scripts and the like where what it does is just what I want. I don'tuse it, though. Instead, I have my own simple netcat-like program(called tcp ) that I use instead.
There are a number of reasons for this, but one significant reasonis that some versions of netcat get the end of file logic wrong. When they see end of file onstandard input, they just close standard input
The processing flow of a network copying program
For my sins, I have dabbled in writing netcat-like programs for sometime, things that take standard input, send it off to somewhere over thenetwork, and write to standard out what they get back from the network.In the process I have formed very definite opinions about how theseprograms should behave in order to be most useful (in scripts and so on),and I feel like writing it down.
For the most part, a network copying program is straightforward; you use select() or
A little script: sshup
(It's been a while since the last little script .)
One of the things I do a fair bit around here is reboot machines. Well, to be morespecific, I reboot machines and then wait for them to come back up sothat I can continue my testing, do more work on them, or verify thateverything is fine. Because I am not crazy I do not do this in themachine room; I do it from my desk.
Waiting for machines to come up and checking periodically
Our current mail system's configuration
A while back I described our old mail system's configuration . Now it's time to describe our current mailsystem's configuration ('current' as of April 2010, although it's beenpretty stable for the past year or two).
Unlike our old mail system, we now trust NFS; we keep /var/mail on our fileservers , along with everythingelse important, and the mail machines that need to deal with it use NFS.This has significantly simplified things.
How not to set up IP aliases on Ubuntu (and probably Debian)
Suppose that you need some IP aliases on an Ubuntu machine. So you go to /etc/network/interfaces and slavishly make yourself some, copying themain stanza a number of times to make entries that looks like this:
auto eth0:0iface eth0:0 inet static address 128.100.1.A network 128.100.1.0 netmask 255.255.255.0 broadcast 128.1
An important thing about how ZFS handles spares
There's an important thing about how ZFS handles spares that's implicitin my first entry , but it's such an important thingthat it deserves to be said explicitly:
ZFS only activates spares when faults happen.
In a traditional spares implementation, spare activation is driven bythe state of the RAID array; if the array is unhealthy and there is aspare, the spare is activated. In ZFS, spares are activated (only) inresponse to fault events; if the right sort of pool fault
The problem with header and footer overlays on web pages
There seems to be a new annoying trend in web page design of makingpersistent headers or footers (or both) that don't scroll out of viewwhen you scroll the web page but instead stay permanently visible atthe top or bottom of your browser window, overlaying the conventionalcontent. Sadly, this design element is a big fat failure (at least forme).
(I'm sure someone has also implemented a sidebar overlay too; I justhaven't seen a website with one yet.)
A DVCS advantage for open source development
Recently, an interesting advantage of DVCSes for open source developmenthas occurred to me: their very nature makes it so that the initialsource of an open source release cannot really reverse itself.
Suppose that you are a company that mightwant to retract and de-release something that has been releasedas open source . With traditionalnon-distributed version control, you could simply shut down the publicsource server for the project; while people who already had copies ofthe source base could in theory put together another public sourceserver