Every so often, someone says something like 'you know, our program has aconfiguration file but also supports runtime reconfiguration via somemagic. Clearly this is wrong, so what we should do is get rid of ourconfiguration file and just make sure the running state is persistent'.If they're feeling nice, they add that the running state will be savedas an XML file.

Every time people say this, sysadmins cry. Here is a very importantthing for real deployments of your program in real environments

Given my earlier entry , here is athesis: some disasters are big enough that you should stop trying torecover rapidly .

The problem with attempting rapid disaster recovery is that significantdisasters are high stress, high pressure situations. Unless you have very good checklists , this is exactly the sort ofsituation where it's easy to have something go catastrophically wrongthrough various situations; missed steps, miscommunication betweenpeople about who was doing what, failing to notice problem indicatorsunder the pressure of driving full speed ahead, interruptions and