The dynamic linking tax on fork()

Most people will tell you that dynamic linking is an unalloyed good, orat least that any effects on performance are small (for simple programsthat don't cascade to a huge set of shared libraries). This isn'tnecessarily so.

A long time ago, back when dynamic linking was new and people weresuspicious of it, I conducted some timings of how dynamic linkingaffected the speed of fork() . To my surprise, the impact wassignificant, and on the hardware of the day it was


Why dynamic linking

Originally, shared libraries and dynamic linking was sold as the way tomake (newly) bloated libraries tolerable: you'd only have one copy ofthe library on disk and in memory, no matter how many Motif applicationsyou had running. (Motif in specific and GUI libraries were really theoriginal poster boys for shared libraries.)

As R. Francis Smith notes in his comment on my previous entry , these reasons are not entirely relevant any more.Disk space is cheap, and (although I haven't


Closures versus classes for capturing state

Possibly like a lot of other people, I have a 'thread pool' Pythonmodule for doing decoupled asynchronous queries of things. The basic interface looks like this:

wp = WorkPool(HOWMANY)for w in what:  wp.enqueue(wrap_up(w))wp.waitDone()res = wp.getanswers()

The important operation is .enqueue( ) , which queues up to be called in a worker thread; its return value is saved, tobe


Making things simple for busy webmasters

It's always nice when people's software saves me from having to wonderif they're up to no good by handing out obvious signs of it. Take, forexample, the spate of people whose web crawling software advertisesitself by having the User-Agent string of:

User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

Evidently no one told them not to stutter. (There are a couple ofvariations in what they claim to


Thinking about project failures

Via a link from MiniMsft 's latest entry ,I wound up reading this item on hiring people. In it I hit an interviewer question I realized that Iprobably wouldn't pass:

6. I will ask you in an interview to describe a project you worked onor effort you made that failed - partially, completely, and miserablyare all OK for response fodder. If you've worked for a substantiallength of time in IT, you've been part of at least one of theseturkeys.


A modular approach to Apache configuration

The traditional way to configure Apache is to take the httpd.conf file shipped with your distribution and start changing it to your localneeds; for example, so that the DocumentRoot and ServerName are right.This is simple and attractive, until you have to upgrade.

The big problem with this approach is that httpd.conf has two sortsof settings; it mixes together settings that you want to customize,like DocumentRoot, with Apache internal things, like AddLanguage andAddCharset. With them all jumbled together, upgrading can be


How not to set up your mail server (part 1)

From our SMTP server logs:

remote from [64.151.68.164]
HELO ADV-RH9-NOCP.YOUR-DOMAIN-HERE.com
554 Unresolvable HELO name: ADV-RH9-NOCP.YOUR-DOMAIN-HERE.com

There is such a thing as taking the instructions too literally.(Or perhaps the error here is in not following instructions; whoknows.)

This appears to be a PLESK-based setup (judging from the


A thought about Technorati

Technorati famously has some problemsindexing blogs .I believe that a lot of these issues may come down to somethingsimple: Technorati's real problem is that it predates the syndicationfeed revolution.

Post syndication revolution blog search engines (like GoogleBlogsearch , Feedster , IceRocket ,and Bloglines ) are actually syndicationfeed search engines . They operate by finding feeds and mining them forthe entries (or at least URLs to the entries, if your feed has partialtext).

But before the syndication revolution there were no widespreadsyndication


Weekly spam summary on March 4th, 2006

It's time for another weekly spam summary. First, let's look at Hotmail,which turns out to be running roughly the same as last week :

  • no messages accepted.
  • 12 messages rejected because they came from non-Hotmail emailaddresses.
  • 49 messages sent to our spamtraps.
  • 2 messages refused because their sender addresses had already hitour spamtraps.
  • 6 messages refused due to their origin IP address (two in theSBL, one in each of the XBL and the CBL, one from Nigeria,

os.walk can be surprisingly slow

Yesterday's tracing made one particular bitof DWiki 's page rendering pop out at me; almost 40% of the time forthe page was going just to create the '(Previous | Next)' links at thebottom.

Because DWiki is entirely filesystem based, its 'metadata' is the stateof the filesystem. Thus, creating the previous and next links requireswalking WanderingThoughts ' directory hierarchy to find out the order ofthe entries. Once I saw the times, I immediately started wondering if I