Every now and then I try a comment spam precaution and it backfires on me. So let me amend myprevious remarks: it turns out that refusing comments from people thatare on the XBL is a bad idea.

It's a superficially attractive idea, which is why I implemented itway back when; the XBL is (theoretically) listing addresses ofcompromised machines and open proxies, and I have seen comment spamattempts from XBL -listed IP addresses. But the XBL itself containswarnings against this sort of usage

In ExploitingPolymorphicWSGI , I talked about how I use the flexibilityof WSGI to run DWiki as either a CGI or a SCGI server by using a smallfrontend CGI program. Because there are some subtle bits to how thisworks, I thought I would write down how the CGI frontend works.

The overall logic is:

• try to talk to an existing SCGI daemon.
• otherwise, check the load and try to start a daemon if the load isbetween a minimum and a maximum, and then talk to it.
• otherwise, if

Every so often, I will hear someone say that Bram Cohen clearly wroteBitTorrent to facilitate piracy (despite any of his claims to thecontrary) because it was deliberately designed to frustrate attemptsto monitor its traffic. This claim irritates me partly because it isclearly wrong, almost blatantly so.

(Disclaimer: I am talking here about classic BitTorrent, as it wasbefore ISPs started whacking things with hammers and people startedreacting.)

There are two important things in a BitTorrent transfer: the peers, thecollection of machines

My minimum technical requirements forreal web spiders are deliberately quite black and white. But there arealso a number of more fuzzy things that I expect from responsible webspiders. Bear in mind that these aren't hard and fast rules and I can'tgive precise numbers and so on.

(As before, this only applies to what I'm calling 'real' or 'legitimate' web spiders ; I can't expectany particular behavior from malicious web spiders.)

Disclaimers in place, here'

Suppose that you have been charged with building a traffic trackingsystem for your NAT gateway to be used when the campus NOC callsyou up to report that your gateway is doing too much of thewrong sort of traffic and gives you the remote IP addressesinvolved. At the same time you don't want to log too much information , so that all this juicy traffic datayou're gathering can't be used to do something like find out all thewebsites someone has been visiting.

No problems; you'

The automounter was more or less created to solve one problem: tryingto avoid having your machine hang when any of the huge list of pokeymachines that exported NFS filesystems that you needed once in a bluemoon went down. Again.

This problem is really an artifact of a much earlier age, of a time whendisk space was so expensive that any machine with any amount of surplusdisk space was pressed into general service. This created a massive webof NFS crossmounts and in the end made everyone's machine depend

Here is a principle of practical system administration:

Later never comes.

Specifically, don't defer something until 'later', because it's nevergoing to happen. Unless you are exceptionally well disciplined and lucky, either you will have lost track of things by the time you havethe free time or you'll discover that any number of other bits nowdepend on the state of the world as it is, and it is no longer at allsimple to change it.

(A surprising amount of practical

I call WSGI polymorphic because with some work, you can run the sameWSGI application in more than one way; all you need is a WSGI adaptorfor the particular environment, whether that is a straight CGI or a SCGIserver.

(Running a WSGI app as a straight CGI might seem strange, but it may beeasier to debug it as a CGI even if you intend to run your applicationin a persistent daemon for deployment. If nothing else, you can be surethat you're not debugging your SCGI

My problem with the term 'web spider' is that I wind up using it to meantwo related but different things, mostly because I don't know of a goodterm for the second one.

One sense of the term is that a web spider is any reasonably autonomousprogram that navigates around the web, poking web URLs, regardless ofwhat it is doing. If you automatically crawl a site and there is not ahuman sitting there supervising you, you're a web spider.

(By this definition