OpenSSH 7.9's new key revocation support is welcome but can't be a full fix

I was reading the OpenSSH 7.9 release notes , as one does, when I ranacross a very interesting little new feature (or combination offeatures):

  • sshd(8), ssh-keygen(1): allow key revocation lists (KRLs) torevoke keys specified by SHA256 hash.

  • ssh-keygen(1): allow creation of key revocation lists directlyfrom base64-encoded SHA256 fingerprints. This supports revokingkeys using only the information contained in sshd(

The needs of Version Control Systems conflict with capturing all metadata

In a comment on my entry Metadata that you can't commit into aVCS is a mistake (for file based websites) ,Andrew Reilly put forward a position that I find myself in somesympathy with:

Doesn't it strike you that if your VCS isn't faithfully recording andtracking the metadata associated with the contents of your files, thenit's broken?

Certainly I've wished for VCSes to capture more metadata than theydo. But, unfortunately, I've come to believe that


Why Prometheus turns out not be our ideal alerting system

What we want out of an alert system is relatively straightforward(and was probably once typically for sysadmins who ran machines).We would like to get notified once and only once for any new alertthat shows up (and for some of them to get notified again when theygo away), and we'd also like these alerts to be aggregated togetherto some degree so we aren't spammed to death if a lot of things gowrong at once.

(It would be ideal if the degree of aggregation


Character by character TTY input in Unix, then and now

In Unix, normally doing a read() from a terminal returns fulllines, with the kernel taking care of things like people erasingcharacters and words (and typing control-D );if you run ' cat ' by itself, for example, you get this line at atime input mode. However Unix has an additional input mode, rawmode , where you read() every character as it's typed (or at leastas it becomes available to the kernel). Programs that supportreadline-


Getting CPU utilization breakdowns efficiently in Prometheus

I wrote before about getting a CPU utilization breakdown inPrometheus , where I detailed building up aquery that would give us a correct 0.0 to 1.0 CPU utilizationbreakdown. The eventual query is:

(sum(irate(node_cpu_seconds_total {mode!="idle"} [1m])) without (cpu)) / count(node_cpu_seconds_total) without (cpu)

(As far as using irate() here goes, see rate


The future of our homedir-based mail server system design

In a comment on my entry on our self-serve system for autoreplies , I was asked a very good question:

Do you think it will ever be possible for you to move to anon-homedir-based mail server at all?

My answer is that I no longer think it would be a good thingfor us to move to a non-homedir-based mail system.

Most mail systems segregate all mail storage and mail processingaway from regular user files. As the commentator is noting, ourmail system


What email messages to not send autoreplies to (late 2018 edition)

Our mail system is very old. Much of the current implementationdates back about ten years, when we moved it to be based on Exim,but the features and in some cases the programs involved go backmuch further than that. One part of it is that we have a localversion of the venerable Unix vacation program, and this localversion goes back a very long time (some comments say it is the 4.3 BSD-Reno version, which would date it to 1990). By


Our self-serve system for 'vacation' autoreplies and its surprising advantage

In the old days, it was just broadly and tacitly assumed thateveryone around here could andwould learn (and use) Unix to get things done in our environment,and so we could provide services purely through traditional Unixcommand line things. This has been less and less true for years,and so there has been a slow drive to provide access to variousservices in ways that don't require logging in and using a shell.One of the traditional painful experiences for people was settingup a vacation or out


rate() versus irate() in Prometheus (and Grafana)

Prometheus's PromQL query language has two quite similar functions for calculating therate of things (well, of counters), rate() and irate() .When I was starting out writing PromQL things, I found people singingthe praises of each of them and sometimes suggesting that you avoidthe other as misleading ( eg , and ).In particular, it's often said that you should use irate() lestyou miss brief activity spikes, which is both true and not truedepending on how exactly


My view on Debian versus Ubuntu LTS for us today

When we started with Ubuntu in2006, Debian was mired in problems such as slow releases andoutdated software that drove people to run 'testing' instead of'stable' . Ubuntu essentially offered 'Debian withthe problems fixed'; Ubuntu LTS had regularly scheduled releases,offered a wide package selection of reasonably current software,and gave us a long support period of five years. This was veryattractive to us and made Ubuntu the dominant Linux here ever since( cf ). However, we don'