Here's an important hint for operators of web spiders:

If I have to touch robots.txt or my web server configuration to dealwith your spider's behavior, the easiest and safest change is to blockyou entirely.

It is therefor very much in the interests of web spider operatorsto keep me from ever having to touch my robots.txt file in thefirst place, because you are not Google .You should consider things like crawl-delay to be desperate lastditch workarounds, not things that you

I've written before about the two sorts of languages , namely the ones where you can write code thatruns as fast as the builtin features, and ones where you can't. Fornow, let me call these 'fast' and 'slow' languages, on the groundsthat it is a slightly less loaded set of terms than other possibleones.

It's recently struck me that one of the effects of this language gapon the 'slow' languages is that it basically cuts off your ability

Here's something important to know about Solaris's MPxIO multipathing : MPxIO identifies disks only by theirserial numbers and identifiers . So if two Solaris devices have thesame serial number, MPxIO concludes that they are two paths to the samephysical disk; it has no actual knowledge of underlying path issues,such as iSCSI target identifiers.

This matters a great deal on iSCSI, because at least some iSCSIinitiators have serial numbers that are set in software. If youaccidentally duplicate some serial numbers between different disks,Solaris

For reasons that are too complicated to fit within the margins ofthis entry, I've recently been dipping into reading some planetblog aggregators. The experience has sparked a realization: planetaggregators have a natural, intrinsic size limit .

The example that crystallized this for me was considering the possiblegrowth of Planet DCS .Between professors and graduate students, there are at least threehundred people here who could haveblogs aggregated onto Planet DCS . If we assume that 200 of themtake seriously the exhortations to blog and

Unix has a number of methods that a suitably privileged process canuse to temporarily assume another UID, do something with those limitedprivileges, and then return back to its old, fully privileged state.The grandfather of these is probably setreuid() , which appeared way back in 4.2 BSD.

Normally, there are any number of ways to examine and interfere withprocesses that are running under your UID; you can send them signals,you can stop them, you can use ptrace() or the equivalent to

When /var was created , people tookeverything in /usr that got written to and just threw it all into onefilesystem. After that, /var became the place that you put anything(besides config files and the like) that needed to change or be writtento, regardless of why.

The problem is that /var has wound up with two very distinct sortsof data in it: private program data and public data. Private programdata is the entire collection of caches, databases, and other trackinginformation

Originally, Unix had no /var ; what is currently put there wentinto /usr instead (with some of it going into /etc ), so you had /usr/log , /usr/spool , /usr/tmp , and so on. Remnants of thisera still linger on in /etc , where you still find a certain number offrequently updated data files like /etc/passwd .

(One might sensibly ask why Unix had both /tmp and /usr/tmp . Myguess is

Here is something that I have not been at all clear about: the sortof checklist usage that I've written about isspecifically using checklists in order to plan and organize one timethings, such as migrating our mail storage .

I care about this because I think of it as a different sort of thingentirely than ongoing work that we do repeatedly and routinely. Ifyou do something routinely and it is not trivial, you should have adocumented procedure for it. However, that procedure may or may notinvolve

One of the things I've been doing much more over the past couple ofyears is using checklists , the virtues of which I'vewritten about before . Recently I have been thinkingabout why they work, and came to the obvious realization: they're aform of talking to the duck .

Before you write down a checklist, you may think that you understandeverything that you need to do, but it is in your head and your head isvery good at fooling you. Like explaining something out