Python likes to handle things through exceptions. As part of this, onUnix it does two important signal changes; it ignores SIGPIPE andcatches SIGINT . Each of these can make a Python program reportapparent errors where a normal Unix command-line program would justsilently exit.

This matters if you want to write a Python program that will play wellin an ordinary command-line environment, alongside things like cat , sed , and awk .

First, people expect that they can ^C a Unix command-line program

Part of the fun of writing programs that deal with HTTP is decodingthings like RFC 2616 to answer somewhat obscure questions about how various thingsinteract. Today's case is the following question:

When can your web server generate a 304 'content not modified'response if it receives a request with both an If-None-Match andan If-Modified-Since header?

If-None-Match and If-Modified-Since are HTTP request headers usedto implement 'conditional GET'

Dating back to our first experiences with msnbot,the MSN Search web crawler, I've known that it was kind of crazy aboutrepeatedly fetching large binary files. Since then, we have pointedthis issue out to MSN Search people more than once and switched to using accurate Content-Types .Recently we had a week of MSNbot not refetching those large binaries,so it looked like MSNbot had finally been fixed.

So much for that. Since 7pm Wednesday night, MSNbot has fetched 3.1gigabytes of various