Some practical notes on the systemd cgroups/units hierarchies
Systemd organizes everything on your system into a hierarchy ofcgroups, or if you prefer a hierarchy of units that happen to beimplemented with cgroups. However, what this hierarchy is (or isgoing to be) isn't always obvious, and sometimes what shows upmatters, for example because you're generating per-cgroup metricsand might hit a cardinality explosion . So here are some noteson things you may see in, for example, systemd-cgls or ' systemctlstatus ' (or if you'
Our varied approaches to upgrading machines with local state
Our normal approach for distribution version upgrades for ourservers is to build a new version of the server on new hardware , then swap it into place by renaming thenew server. Sometimes, as for our SLURM cluster ,we have sufficiently cattle-like machines that we will take one or a few of them out of service and reinstallthem in place. All of this is straightforward for machines thathave basically no local state, but this isn't the case for all ofour machines. Servers like our ZFS-
More use of Rust is inevitable in open source software
Recently, I saw a poll on the Fediverse about making Rust a harddependency for fwupd .This got me to post a lukewarm take of my own :
Lukewarm take: the spread of Rust in open source software isinevitable, because nothing else fills the niche for 'C/C++ butstrongly safe'. We need a replacement C because in general we can'twrite safe C/C++ at scale.
Rust isn't really my thing , but this shift impliesthat sooner or later
Sorting out PC chassis power switches for ATX power supplies
My problematic home desktop continuesto be problematic, so recently over on the Fediverse I had aclever idea about 'replacing' the case front panel with just astand-alone ATX chassis power switch tucked away somewhere , which @penguin42suggested improving by relying on setting the BIOS to 'always poweron when AC is restored' and not having a chassis power switch at all . This leftme with a certain amount of curiosity about ATX chassis powerswitches and the matrix of possibilities for what you can do to
The systemd journal is primarily time-based
Recently, I encountered a surprising systemd behavior :
Dear systemd: apparently using absolute system date (at boot?!)to determine 'the system boot before this one' and so on is notnecessarily a great idea, given that systems can boot with scrambledsystem time. If I want to look at 'the previous boot', I want that towork even given bad time. Especially if the system booted with badtime, ran with bad time for a bit, and then rebooted again.
(There are
Handling numbers in Vim when they have a dash in front of them
Over on the Fediverse, I mentioned a Vim situation I'd run into :
Here's a situation Vim's Ctrl-A feature for handily incrementingnumbers doesn't readily handle: if you have machines called eg'something-1' and want to make that 'something-2', 'something-3',and so on. Vim will interpret the '-1' bit as the number -1 andauto-increment it to 'something0'. There is probably a clever
The power of URLs you can use with query parameters and a HTTP GET request
I recently wrote about some aspects of my dmenu setup , including using a custom$PATH that contains a bunch of little utility scripts . These little scripts are animportant part of making my dmenu setup so useful and a crucialbuilding block of my environment , but inturn a lot of them are enabled by something else. In practice, alot of what I do with dmenu is to open a Firefox window on somesort of URL, and in turn this is relies on being able to createURLs that do useful
The Prometheus cardinality issues with systemd unit-related metrics
Over on the Fediverse I said something about cAdvisor's Prometheusmetrics :
Unless I'm missing something, cAdvisor's Prometheus metrics exporthas no way to limit what cgroups it generates metrics for. This isfatal on systems used by users, since it leads to a label cardinalityexplosion as the cgroups reported on will include 'session-NNN.scope'cgroups, with a constantly increasing and basically never re-used NNN.
This is a general issue for any Prometheus metrics that are relatedto particular systemd units
Detecting missing or bad Go modules and module versions
I recently wrote about the case of a half-missing Go import , where a particular version of a module hadbeen removed upstream but was still available through the (default)Go module proxy , so things didn't reallynotice. This raises the interesting question of how you find thissort of thing, in all of at least three variations. Unfortunately,right now there are no great answers, but here's what I can see.
The most straightforward case is if a module has been marked asentirely
Systemd unit templates don't provide a native way to have multiple parameters
We have a collection ofApache servers, and as part of our Prometheus metricssystem we indirectlyscrape metrics from them, using a third party Apache exporter that queries Apache's mod_status information and converts it to Prometheus metrics. Because the exporter'sauthor chose to write it that way, you need to run one instance of theexporter per Apache server you want to extract metrics from. Ordinarypeople would probably run these exporter instances on the Apache hoststhemselves. We opted to instead run all of the exporters