Our backup MX server was easy to build, but yours might not be

I recently mentioned that we'd built a backup MX server due to concerns prompted by a scheduled power outage . In a comment on that entry, Greg A. Woods said something that I broadly agree with:

I think backup MX hosts are, generally speaking, a bad idea in modern times (even going back a couple of decades).

[...]

The added maintenance overhead and headache of keeping a full-time backup MX host running and reliably forwarding ALL email it collects, and reliably rejecting all email it should reject, isn't usually worth the bother.

One reason that we implemented a backup MX is that this isn't our experience. Our backup MX was easy to build and is essentially trivial to keep in reliable operation. However, this isn't because we have some special trick to running backup MXes; instead, it's because we have a general mail architecture that enables it.

Many, many years ago we moved from a mail architecture that was essentially monolithic to one that had an external MX gateway that was stuck in front of our central mail server. This transition involved creating what I call a 'white-box' mailer environment , where knowledge of things like valid local addresses and domains was materialized in text files and reusable in many contexts. Our spam and virus filtering is also done with FOSS components , which we can more or less run as many copies of as we like.

So our backup MX is essentially a clone of our regular external MX gateway machine, except that it has the MTA and the anti-spam stuff on the same machine (and we may do this for the next version of the external MX gateway, now we know more about how much load the anti-spam stuff creates). The backup MX server uses the same white-box mail information that our external MX gateway machine does, and we arranged for it to sit in a network environment where it could deliver accepted mail straight to our central mail server (instead of later delivering it to the normal external MX gateway, which would have added more hops and more redundant spam checking).

(All of the changes from the regular external MX gateway were things that we already had in operation on other machines and needed only modest tweaks to deal with the unique parts of this one.)

This is only possible because we already had all of the pieces. We have a general framework for installing and operating servers, we had an external MX gateway separate from the main mail system, that external MX gateway didn't rely on internal services to do things like validate addresses, and we didn't have commercial software involved that might have had license restrictions that prevented us from running an extra copy on our new backup MX.

We're also making life easier on ourselves by only running this backup MX temporarily, and with a configuration for valid email addresses, spam settings, and so on that is effectively frozen because all of the machines and services that could change any of that are powered off. That way we don't have to worry about what happens if the network connection between the backup MX and us gets blocked and the backup MX starts drifting out of sync on what email addresses are valid and so on.

If we hadn't already moved from a monolithic black-box mailer environment to a multi-machine white box one , building and running a backup MX host would have had all of the issues that Greg A. Woods identified. The existence of some of these issues is part of why spammers like to probe your backup MX . Also, in general I still agree with my old entry on the case against a full time backup MX , although modern email makes me nervous about the potential for aggressive mail delivery timeouts .

(In my old terminology , what we've built is technically a redundant MX. But that's a happy accident of the available network connectivity where this machine is going to be located for the power outage, and it could have had to deliver mail to our regular external MX gateway.)