The marshal module looks interesting for persisting and retrieving lightweight data,but the big question to me has always been whether in exchangefor constraining your data down to simple structures of primitivetypes you got something that was actually faster than the cPicklemodule .

So today I decided to finally answer the question by doing sometiming tests. I won't claim that these are comprehensive or entirelyscientific, but I do have some results:

• the speed difference is mostly in dumping things; marshal andcPickle generally load things as fast as

As a follow-up to 64BitDrawback : after we had several machines crashdue to being driven out of memory, dealing with the whole issue suddenlygot a whole lot more urgent and we opted to try to solve it by turningon Linux's strict overcommit mode for swap allocation. At first we didthis only on our compute servers, but after some of our login serversalso OOM'd and crashed, we enabled it on them too.

(Strict overcommit has the great advantage that we don'

Before I can talk about more interesting mailer things, I have toexplain how our old mail system was configured.

Our old mail system makes perfect sense once you realize that it wasmore or less designed around the idea that nothing should ever haveto be done over NFS. In order to manage this, each different sort ofprocessing had to be done on the machine that held the relevant files;deliveries to /var/mail were done on the postbox machine, which had /var/mail on local disks,

Many systems seem to be not really designed for a long term storagemanagement environment . Instead they seem toopt for a kind of planned obsolescence approach where they assume thatyou will buy them, run them more or less into the ground without reallychanging or upgrading anything, and then replace them wholesale in abig, painful, user-visible bang.

From the perspective of a long term storage management environment this is a crazy thing to do; with no growth and thus no future, thesesystems are basically closed boxes