My minimum technical requirements forreal web spiders are deliberately quite black and white. But there arealso a number of more fuzzy things that I expect from responsible webspiders. Bear in mind that these aren't hard and fast rules and I can'tgive precise numbers and so on.

(As before, this only applies to what I'm calling 'real' or 'legitimate' web spiders ; I can't expectany particular behavior from malicious web spiders.)

Disclaimers in place, here'

Suppose that you have been charged with building a traffic trackingsystem for your NAT gateway to be used when the campus NOC callsyou up to report that your gateway is doing too much of thewrong sort of traffic and gives you the remote IP addressesinvolved. At the same time you don't want to log too much information , so that all this juicy traffic datayou're gathering can't be used to do something like find out all thewebsites someone has been visiting.

No problems; you'

The automounter was more or less created to solve one problem: tryingto avoid having your machine hang when any of the huge list of pokeymachines that exported NFS filesystems that you needed once in a bluemoon went down. Again.

This problem is really an artifact of a much earlier age, of a time whendisk space was so expensive that any machine with any amount of surplusdisk space was pressed into general service. This created a massive webof NFS crossmounts and in the end made everyone's machine depend