The Prometheus host agent's CPU utilization metrics can be a bit weird
Among other metrics, the Prometheus host agent collects CPU timestatistics on most platforms (including OpenBSD, although it's notlisted in the README). This is the familiar division into 'usertime', 'system time', 'idle time', and so on, exposed on a per CPUbasis on all of the supported platforms (all of which appear to beprovided with this by the kernel on a per-CPU basis). We use thisin our Grafana dashboards, in two forms.
OpenBSD's 'spinning' CPU time category
Unix systems have long had a basic breakdown of what your CPU (orCPUs) was spending its time doing. The traditional division is usertime, system time, idle time, and 'nice' time (which is user timefor tasks that have their scheduling priority lowered through nice(1) or the equivalent),and then often 'interrupt' time, for how much time the system spentin interrupt handling. Some Unixes have added 'iowait' , which is traditionally defined as 'thesystem was idle
Any KVM over IP systems need to be on secure networks
In response to my entry wishing we had more servers with KVM overIP (among other things) now that we're working remotely , Ruben Greg raised a very important issuein a comment:
KVM over IP: Isnt this a huge security risk? Especially given the rareupdates or poor security of these devices.
This is a very important issue if you're using KVM over IP, for tworeasons. To start with, most KVM over IP implementations have turnedout to have serious security issues, both in
The problem of your (our) external mail gateway using internal DNS views
Suppose, not hypothetically, that you have an external mail gateway(your external MX, where incoming email from the Internet is handedto you). This external MX server is a standard server and so youinstall it through your standard install process . As part of that standard install,it gets your normal /etc/resolv.conf , which points it to yourlocal DNS resolvers. If you have a split horizon DNS setup , your local, internal DNS resolvers willnaturally provide the internal view, complete with internal
How we set up our ZFS filesystem hierarchy in our ZFS pools
Our long standing practice here ,predating even the first generation of ourZFS fileservers , is that we havetwo main sorts of filesystems, home directories ( homedir filesystems)and what we call 'work directory' ( workdir ) filesystems. Homedirfilesystems are called /h/NNN (for some NNN) and workdir filesystemsare called /w/NNN; the NNN is unique across all of the differentsorts of filesystems. Users are encouraged to put as much stuff aspossible in workdirs and can have as many of them
Why we use 1U servers, and the two sides of them
Every so often I talk about '1U servers' and sort of assume thatpeople know both what '1U' means here and what sort of server Imean by this. The latter is somewhat of a leap, since there are twosorts of server that 1U servers can be, and the former requiressome hardware knowledge that may be getting less and less commonin this age of the cloud.
In this context, the 'U' in 1U (or 2U, 3U, 4
Avoiding the 'dangling else' language problem with mandatory block markers
There is a famous parsing ambiguity in many programming languages knownas the dangling else ,where it's ambiguous which ' if ' statement an ' else ' is associatedwith. In C, for example, you can write:
if (a)if (b) res = 10;else res = 20;
Which if the else is associated with is somewhat ambiguous; isit the first or the second? (C provides an answer, as any languagethat allows this must.)
It's
Wishing for a remote resilient server environment (now that it's too late)
Due to world and local events , we have all abruptly been workingfrom home and will be for some time. The process has made me wishfor a number of differences in our server environment to make itwhat I'll call more remote resilient , more able to cope with lifewhen you don't have sysadmins down the hall during the hours ofofficial support.
One big ticket thing I now wish for is some kind of virtual machinehost for test machines, one that can be used remotely over somecombination
Sorting out Go's 'for ... = range ..' and when it copies things
I recently read Some tricks and tips for using for range in GoLang ,where it said, somewhat in passing:
[...] As explained earlier, when the loop begins, it will copy theoriginal array to a new one and loop through the elements, hence whenappending elements to the original array, the copied array actuallydoesn't change.
My eyebrows went up because I'd forgotten this little bit of Go,and I promptly scuttled off to the official specification to read and understandthe details
Make sure to keep useful labels in your Prometheus alert rules
Suppose, not entirely hypothetically, that you have some metrics thatare broken out across categories but what you care about are the totalnumber of things together. For example, you're monitoring some OpenBSDfirewalls and you care about the total number of PF states, but yourmetrics break them down by protocol ( this information is availablein ' pfctl -ss ' output ). So youralert rule is going to be something like:
- alert: TooManyStates expr: sum( pfctl_protocol_entries ) by (