A spate of somewhat alarming flaky SMART errors on Crucial MX500 SSDs

We've been running Linux's smartd on all of our Linux machines for a long time now, and over that time it'sbeen solidly reliable (with a few issues here and there, like not alwayshandling disk removals and (re)insertions properly). SMART attributes themselves may ormay not be indicative of anything much, but smartd does reliably alerton the ones that it monitors.

Except on our new Linux fileservers . For a significant amountof time now, smartd has periodically been sending us


Why I'm usually unnerved when modern SSDs die on us

Tonight, one of the SSDs on our new Linux fileservers died. It's not the first SSDdeath we've seen and probably not the last one, but as almost always,I found it an unnerving experience because of a combination of howour SSDs tend to die, how much of a black box they are, and how they'resolid-state devices.

Like most of the SSDs deaths that we've had, this one was very abrupt;the drive went from perfectly fine to completely


Firefox, WebExtensions, and Content Security Policies

Today, Grant Taylor left some comments on entries here (eg yesterday's ).As I often do for people who leave comments here who include theirhome page, I went and visited Taylor's home page in my usual Firefoxbrowser, then when I was done I made my usual and automatic Foxygestures gesture to close the page. Except that this time, nothing happened(well, Firefox's right mouse button popup menu eventually came upwhen I released the mouse button).

In the old days of


Link: HTTPS in the real world

Robert Heaton's article HTTPS in the real world ( via ) is aboutthe difference between HTTPS in theory, in the cryptographic worldof Alice and Bob, and HTTPS in practice, in the messy real worldwhere CAs cannot be fully trustedand people lose their keys and so on. To pick one little bit toquote:

[...] But the real world has still managed to piece together a veryserviceable public-key cryptography system by patching over theholes and omissions and naivety of the introductory world


Modern Bourne shell arithmetic is pretty pleasant

I started writing shell scripts sufficiently long ago that the onlyway you had to do arithmetic was to use expr . So that settled inmy mind as how you had to do arithmetic in shell scripts, and sinceusing expr is kind of painful (and it has an annoying, obscuremisfeature ), mostly I didn't. If Iactually had to do arithmetic in a shell script I might reach forrelatively heroic measures to avoid expr , like running numbersthrough awk . In one relatively recent occasion, I had


Why we like HTTP Basic Authentication in Apache so much

Our web server of choice here is Apache, and when we need some sort of access control for it forpeople, our usual choice of method is HTTP Basic Authentication (also MDN ).This is an unusual choice these days; most people use much moresophisticated and user-friendlier schemes, usually based on cookiesand login forms and so on. We persist with HTTP Basic Authenticationin Apache despite this because, from our perspective, it has threegreat advantages.

The first advantage is that it uses a username and


Some basic ZFS ARC statistics and prefetching

I've recently been trying to understand some basis ZFS ARC statistics,partly because of our new shiny thing and partly because of a simplemotivating question: how do you know how effective ZFS's prefetchingis for your workload?

(Given that ZFS prefetching can still run away with useless IO , this is something that I definitely want tokeep an eye on.)

If you read the arcstat manpage or look at the raw ARC kstats, you'll very soon notice things like'prefetch hits percentage' arcstat


The brute force cron-based way of flexibly timed repeated alerts

Suppose, not hypothetically, that you have a cron job that monitorssomething important . You want to benotified relatively fast if your Prometheus server is down, so yourun your cron job frequently, say once every ten minutes. However,now we have the problem that cron is stateless ,so if our Prometheus server goes down and our cron job startsalerting us, it will re-alert us every ten minutes. This is toomuch noise (at least for us).

There's a standard pattern for


Linux disk IO stats in Prometheus

Suppose, not hypothetically, that you have a shiny new Prometheus setup and you are running the Prometheushost agent on yourLinux machines, some of which have disks whose IO statistics mightactually matter (for example, we once had a Linux Amanda backupserver with a very slow disk ). ThePrometheus host agent provides a collection of disk IO stats, butit is not entirely clear where they come from and what they mean.

The good news is that the Prometheus host agent gives you the rawLinux kernel disk statistics


Wget is not welcome here any more (sort of)

Today, someone at a large chipmaker that will go unnamed decided(or apparently decided) that they would like their own archivedcopy of Wandering Thoughts . So they did what one doeshere; they got out wget ,pointed it at the front page of the blog, and let it go. I was luckyin a way; they started this at 18:05 EST and I coincidentally lookedat my logs around 19:25, at which point they had already made around3,00