People might want to think about saving a copy of Go 1.17

The news of the time interval is that Go 1.18 has been released . There are a number of famous bigchanges in Go 1.18; obviously generics, but I think that officialfuzzing support and workspaces are likelyto have a bigger impact in the near future (certainly I think thatmore people should use fuzzing than should touch generics rightnow). But there's another important change, which is that modulemode is now mandatory ,although the release notes don't directly say that


How Linux dynamic loaders can be both shared objects and executables

I was reading RPATH, or why lld doesn’t work on NixOS ( via )and ran across this comment:

[...] And at runtime loader (also known as dynamic linker (its binaryname is something like ld-linux-x86-64.so, but despite the .sosuffix, it’s an executable (I kid you not, this stuff is indeedthis confusing))) loads the executable together with shared librariesrequired by it. [...


Where cut comes into Unix (and a bit on the history of awk)

The cut command is in some ways one of those little Unix oddities, becausein many ways (although not all of them) it duplicates the functionalityof awk . Both commands have been part of my Unix landscape forlong enough that I don't think about where they come from, but todayI wound up curious about cut 's history.

Awk famously comes from V7 Unix , and isone of the signature Unix programs introduced there (see theWikipedia entry for more). Bycontrast, cut comes


We do see ZFS checksum failures, but only infrequently

One of the questions hovering behind ZFS is how often, in practice,you actually see data corruption issues that are caught by checksumsand other measures, especially on modern solid state disks. On ourold OmniOS and iSCSI fileserver environment we saw somewhat regular ZFS checksum failures, but that environmenthad a lot of moving parts, ranging from iSCSI through spinning rust. Our current fileserver environment uses local SSDs, and initially it seemed we were simply notexperiencing checksum failures any more. Over time, though, we haveexperienced some


I wish ZFS pools kept a persistent count of various errors

Famously, ZFS pools will report a count of read, write, and checksumerrors on the level of the pool, vdevs, and individual devices, countsthat are persistent over reboots (and thus pool exports and imports).Equally famously, ZFS expects you to clear these counts when (and if)you resolve problems; for example, if you want to see if you have apersistent checksum problem or a one-time thing, you'll normally clearthe error count and re-scrub the pool.


Filesystems can experience at least three different sorts of errors

Yesterday I wrote about how it would be nice if Linux exposed a countof disk errors and mentioned thatsome Linux filesystems do expose such a count of errors, but it's notclear what sort of errors they mean. This sounds like a peculiar thingto say, but in fact filesystems can experience at least two or threedifferent sorts of errors. I will call these I/O errors , integrityerrors , and structural errors .

An I/O error happens when the underlying storage device returns anerror from


It would be nice if Linux had a count of disk errors in sysfs

The Linux kernel knows when disk IO errors happen and it will tellyou about them (at sometimes copious length, see 1 , 2 , 3 ). But only in the kernel message log,in a format that can vary from disk type to disk type and changesover time. This makes it pretty hard to notice when disk errorshappen somewhere out there in your fleet and to keep track of themover time.

Given that the Linux kernel knows about these errors, it would benice if it kept a per


Linux disk names you can encounter in your Prometheus host metrics

We recently added our first server with NVMe drives to our fleet. Naturally we hooked itup to our Prometheus setup so it would show up on our Grafana dashboards, including thedashboards for disk IO. This turned up a little issue, which isthat Linux names NVMe drives very differently than SATA (and SAS)drives . Our dashboards had previously been lookingonly at Prometheus disk metrics for devices with 'sd.*' names,so they needed some changes to add nvme.* names. This set meto


Hardware can be weird, server and USB keyboard edition

Over on Twitter, I said some not entirely justified nasty thingsabout the Ubuntu 20.04 server install ISO , becauseit wasn't letting me switch to an alternate Linux virtual consoleto get a shell so I could see what was going on with some things. The (current) Ubuntu documentation certainly impliesthat they do things differently, but that turned out not to be quiteit. Instead, what was going on was an interesting and odd interactionbetween the keyboards I tried and the server. Since


The convenience of multi-purpose monitoring (in Prometheus)

Recently I mentioned to someone that our TLS certificate expiryalerting is very convenient in that we mostlydon't have to specifically do something to monitor TLS certificateexpiry. Instead, we get it for free when we start monitoring a servicethat uses TLS. For example, if we add a check for a new HTTPS websitehere being up properly, that automatically adds TLS certificate expirymonitoring.

The specific mechanics of this are that the Prometheus Blackboxexporter thatlets you do external health checks of services also exposes variousmetrics for