We've permanently disabled overlayfs on our servers

I tweeted :

Oh look, yet another Linux kernel local exploit in the overlayfsmodule. Time to permanently blacklist it on all of our machines.

Today's bugs are CVE-2016-1576 and CVE-2016-1575 ( via ).There have been others before ,and probably more that my casual Internet searches aren't turningup.

Based on my experiences so far, the two most common ingredients inexploitable kernel security issues we've


The university's coordination problem

In response to my entry on how we can't use Let's Encrypt inproduction , Jack Dozier lefta comment asking if we'd looked into InCommon's Certificate Service . InCommon is basically aconsortium of US educational institutions that have gathered togetherto, among other things, create a flat cost CA service; apparently,for $15k US or so a year, your university can get all the certificatesyou want (including for affiliated organizations). This soundsgreat, but at least here


Why the Ubuntu package update process makes me irritated

We have fifty or so Ubuntu machines in our fleet, which means thatwhen Ubuntu comes out with package updates we have to update themall. And this is rather a problem. First, obviously, fifty machinesis too many to do updates on by hand; we need to automate this.Unfortunately Ubuntu (or Debian) has made a series of decisionsabout how packages and package updates work that cause significantamounts of pain here.

(Although Debian and Ubuntu share the same package format, I don'thave


My two usage cases for Let's Encrypt certificates

As I mentioned yesterday , we unfortunatelycan't use Let's Encrypt certificatesin production here . That doesn'tmean I have no use for LE certificates, though. Instead I have twodifferent ones.

My first usage case for LE certificates is as the first stop fortemporary certificates for test machines at work. I not infrequentlyneed to set up test versions of TLS-based services for variousreasons, including testing configuration changes, operating systemupgrades, and even whether or not I can make some random


We can't use Let's Encrypt on our production systems right now

I really like Let's Encrypt , the newfree and automated non-profit TLS Certificate Authority. Free ishard to beat, especially around here , and automatically issuedcertificates that don't require tedious interaction with websitesare handy. And in general I love people who're striking a blowagainst the traditional CA racket. Unfortunately, despite all ofthat, there's basically no prospect of us using LE certificatesin production around here .

The problem is not any of the traditional ones you might think of


Two models of dealing with cookies in Firefox with addons

Recently on Twitter, Dan McDonald was looking for a Firefox cookiehandling addon .I had some opinions on this but Twitter being Twitter and me being meI wasn't entirely articulate about them at the time. So here is myattempt to do it better.

There are at least two fundamental models for dealing with cookies(in Firefox and probably elsewhere). The first model is to not allowcookies into your browser session at all. You default-deny allcookies (even first party ones) and whitelist only


The many load averages of Unix(es)

It turns out that the meaning of 'load average' on Unixes is rathermore divergent than I thought it was. So here's the story as I knowit.

In the beginning , by which I mean 3 BSD, theload average counted how many processes were runnable or in shortterm IO wait (in a decaying average). The BSD kernel computed thiscount periodically by walking over the process table; you can seethis in for example 4.2BSD's vmtotal() function .Unixes


Whether or not to use cgo for Go packages, illustrated in a dilemma

Recently Dave Cheney wrote cgo is not Go where he very stronglyadvocates for not getting functionality into Go by using cgo to interfaceto existing C libraries. More directly, he writes:

I believe that when faced with reimplementing a large piece of C codein Go, programmers choose instead to use cgo to wrap the library,believing that it is a more tractable problem. I believe this is afalse economy.

He goes on to run down a laundry list of problems that using cgocauses. All of them are real


SMTP submission ratelimits should have delays too

We've recently enabled ratelimits on our mail submission servers . By this I mean that when youhit the ratelimits (which are applied to recipients, ie RCPT TO commands), the mailer gives you 4xx replies and your client willprobably error out. But when I set up the ratelimits, I didn't justdo that; I also made the mailer delay for 10 seconds before respondingto every ratelimited RCPT TO . It's my view that this is a usefuland in fact important thing to


Your outgoing mail system should have a per-sender stop switch

Here is something important we have come around to realize as oneresult from recent events .Put simply, every system that handles outgoing user-generated emailshould have some method to immediately stall and stop all emailfrom a specific user. You want this for the obvious reason; whenyou discover you have a compromised user account that's being usedto send spam, you can immediately stop just their email instead ofhaving to take down your entire outgoing email system.

When you implement this, don't just implement