Efficiently distributing huge files to lots of workstations

CQUEST 's workstations have a pieceof software that needs a read only 5+ gigabyte set of datafiles. Naturally we put the dataset on the local workstation disks,which works great. (These days it's hard to buy machines with lessthan 40 gigabyte disks, which leaves about 30 gigabytes free even witha generous Linux installation. We don't put user data on workstationsfor all the obvious reasons.)

However, every so often the dataset changes and we need to propagateall 5


Why I really dislike the Singleton design pattern

To quote Glyph Lefkowitz (from here ):

Aah, the singleton. Global variables for the new millennium.

I'm not against global variables, but I am against misleading peopleabout them. The Singleton pattern lies about them; it looks like youhave a normal object, but you really have a global variable withoutknowing it.

(And if everyone knows you have a faux global variable, you are justtarting up a global variable in 'object-oriented' clothes. This islike respecting the letter of


The problem of being overcautious

Today's fire drill was caused by our printing system not printing;since it is the first day of classes, it was not a good time todiscover this. After fixing a couple of small problems, the bigstumbling block was authentication not working.

Our printing system has a central machine that handles quotamanagement and a per-lab machine that handles the actual printspooling and printing. This requires the labmasters to talk to thequota server to tell it about pages that got printed.

Because I am paranoid,


Concurrency is tricky

Concurrency may or may not be hard (I know people who disagree with meon that). But I am sure that it is tricky. As an illustration, I justfixed a DWiki concurrency bug that I first spotted in MyFirstCommentSpam .

For simplicity, DWiki stores each comment as a file in a directoryhierarchy that mirrors the page's DWiki path; if you comment on the DWiki page /foo/bar , the comment will be a file in a /foo/bar/ directory (under a separate top


(什麼時候懂得汲取教訓?!)

1. [ 刪97字 ]

2. 民間博物館計劃: 整整一條利東街

集成展廊
灣仔集成中心高層商場(軒尼詩道302-308號)
展期:9月10日至10月9日,1100~2000

我地沉迷現代詩之輩,多喜用「完整」,嫌「整整」

Things not do for your network daemon's debugging option

It's always nice when people build debugging support into theirnetwork daemon. However, I have a modest suggestion:

When your daemon is designed to be run under inetd / xinetd ,writing debugging output to standard error is a bad plan.

(You win bonus points for also logging it to syslog, so that aninnocent person might be deluded into thinking that it's only going via syslog.)

Because, you see, all that nice debugging output is going to go flyingstraight off to the clients talking


The annoyance of arbitrary limits

In theory, WanderingThoughts has been syndicated on LiveJournal as the LJ usercks_techblog( here ) forsome time. In practice, LiveJournal has an undocumented, hard-codedlimit on how large syndication feeds can be; go over the limit and LJrefuses to process your feed. Because I dislike small limits, DWiki defaults to putting lots of entries in syndication feeds, more thanenough to exceed LiveJournal's size limit given how I write.

(LiveJournal compounds the problem by not allowing a syndication feedto be


病晴雯勇補雀金裘

迪士尼今日開幕。點病都要賀一賀佢。

1.
我係旺角人、多年無牌小販忠實顧客,一向都唔多鍾意食環署。食環署向以對無牌小販苛刻而惡名昭著,不過做政府心理輔導的朋友說,食環署的人工作和心理壓力最大,很多人間慘劇

Comment spam writ large

This Friday I discovered a neglected web-based bulletin board on oneof our web servers that was open for posting. Unfortunately, commentspammers had discovered it months before I did and had been gleefullyexploiting it since then. The result gives me an unpleasant, fullthrottle view into the world of comment spammers.

The raw numbers are appalling: in the time they were active, thecomment spammers posted at least 233,799 spam comments (fortunately,the web board only stored the last 10


Weekly spam summary on September 10th, 2005

Overall connections are up from last week : 239,000 SMTP connections from39,000 different IP addresses. The SMTP frontend's highwater mark isup again, hitting 29 simultaneous connections.

Top 10 kernel level SMTP rejections:

Host/Mask           Packets   Bytes213.4.149.11          13913    638K192.35.251.3          13025    625K