Using Prometheus subqueries to look for spikes in rates
One of the things that people traditionally want to do with graphsof things like network bandwidth or disk IO rates is to look forbrief spikes in usage (or brief dips, depending on what you'rehunting for). The easy case of this is when the resolution of yourgraph is high enough that it can display the instantaneous rate atevery metric point; this will always capture and display both spikesand dips. When the resolution of your graph is not this high, thefirst thing you need to do
A surprisingly arcane little Unix shell pipeline example
In The output of Linux pipes can be indeterministic ( via ),Marek Gibney noticed that the following shell command has indeterminateoutput:
(echo red; echo green 1>&2) | echo blue
This can output any of "blue green" (with a newline between them),"green blue", or "blue"; the usual case is "blue green". Fullyexplaining this requires surprisingly arcane Unix knowledge.
The "blue green" and "green blue" outputs are simply a schedulingrace
Really understanding diffs requires knowing their context too
A lot of things like presenting changes in the form of diffs(generally unified diffs these days, fortunately). Some of the timewhen I'm reading these diffs in various contexts ( eg ), I've felt that I was struggling tofully understand what I was seeing. Today, I had a realization aboutthis that feels completely obvious in retrospect, namely that understanding diffs relies on implicit contextual knowledge oftheir surrounding code . Most of the time, to understand the changethat a diff is making you
Understanding a change often requires understanding how the code behaves
Yesterday I wrote about how really understanding diffs requiresknowing their context too . An additional partof the problem is that you can't understand most changes in isolation,just as a change. To actually understand a change, you usually needto be able to compare the before and after behavior of the code,and in order to do that you must understand (or work out) what thatbehavior is.
This is a big part of why you have to reconstruct those before andafter versions of the code in your
What you get when you do a DNS A record lookup for a CNAME'd name
One of the things that we do with our monitoring system is to checkthat a variety of DNS servers can resolve things from our university subdomain, includingour own authoritative server, the external secondary NS, our internalresolving servers, and so on, and recently we started checkingthrough Google's and Cloudflare's public servers. The specifichostname I picked to check the address of is actually a CNAME, sorecently I wound up asking the obvious question: if you look up theA record for something that is
Taking advantage of the Linux kernel NFS server's group membership cache
Yesterday I wrote about looking at and flushing the NFS server'sgroup membership cache , whose currentcontents are visible in /proc/net/rpc/auth.unix.gid/content . Atthe time I was simply thinking about how to manage it, but afterwardit struck me that since it can get reasonably large, the groupmembership cache will tell you some potentially quite valuableinformation. Specifically, the group membership cache will oftentell you who has used your NFS server recently .
Every time an NFS(
How to see and flush the Linux kernel NFS server's group membership cache
One of the long standing limits with NFS v3 is that the protocolonly uses up to 16 groups . In order toget around this and properly support people in more than 16 groups, various Unixes have various fixes .Linux has supported this for many years ( since at least 2011 )if you run rpc.mountd with -g aka --manage-gids . If you douse this option, well, I'll just quote the rpc.mountd manpage :
Accept requests from
Using Prometheus subqueries to do calculations over time ranges
Subqueries are a new feature in Prometheus 2.7. Their usual use is to nesttime range queries, such as a max_over_time of a rate , ascovered in, for example, Brian Brazil's How much of the time ismy network usage over a certain amount? .However, they can be used in another, perhaps less obvious way.Put simply, subqueries let you use time based aggregation onexpressions .
Suppose, for example, that you are collecting basic NTP informationfrom your
Link: Vim anti-patterns
Tom Ryder's Vim anti-patterns is nominallyabout anti-patterns but it's really about teaching various better,more advanced ways to do things in vim. For each anti-pattern, Rydershows a positive pattern (or several of them), and they're worththinking about and maybe adopting. If nothing else, I learned newbits of vim.
(This is from 2012 and I believe I read it then, but it's worthre-reading every so often
ntpdate has a surprising restriction on what it will sync to
One of the things that we monitor these days is the health of ourthree local OpenBSD NTP servers that all of our machines get timefrom. For various reasons we do it using ntpdate , partly because that's what we synchronize time with these days .Last Wednesday, we started getting alerts that ntpdate couldn'tsynchronize to them; first to one server, and then expanding to asecond one of the three.
When I started investigating I expected to find that the OpenBSDNTP daemon had fallen over or lost