Today, Ben Cotton tweeted :

When I see a #sysadmin say "I don't use any configuration management",I mentally add "because I'm allergic to competence."

I have a complex reaction to things like this and after a small conversation I condensedsome of my opinions to this:

@thatcks :My overall view on #sysadmin automation is that automation addscomplexity as well as removes it. How each side balances out varies.
@thatcks :Before you automate, you manage your systems.

This is both a sysadmin war story and a great example of what I'll calla full-stack problem.

One of our fileservers is significantlymore problematic than all of the others, for reasons that we don'tunderstand. One of the ways that it was problematic was that when itrebooted, it would connect to iSCSI target disks only very slowly.Solaris prints a line for every configured iSCSI target ( which can bea problem ), and we'd usually see these lines going by ata

As a result of recent events ,I've built up a collection of DTrace scripts for monitoring andreporting on our fileserver environment , wherewe use NFS v3 on top of ZFS on top of iSCSI. Since I grumbledearlier about the lack of easily findable DTrace scripts for this, I'vemade our scripts public on Github as siebenmann/cks-dtrace (where you can read moredetails about what's there). They're written for Solaris 10 update 8(plus some patches)

After discovering something attempting some open relay checks , I got curious about whether this was a one-off orif there were clear signs of other open relay checks. To give you aspoiler, the answer is that I can't completely tell because there isa bunch of noise in my data (and on top of that I'm not sure how toanalyze it), but it seems possible.

What I can easily get from Exim's logs is triples of IP address, MAILFROM , and

If only so that I never have to carefully reinvent this code again,here is how I'm doing randomized read IO to avoid ZFS prefetching . Since ZFS prefetching is the mostsuperintelligent form of prefetching I've ever seen, I expect that thisapproach would also avoid prefetching on other filesystems and OSes.

The following code is in Python and assumes you have a readat() function that does the basic read (and also does whatever timetracking and so on you want):

KB = 1024

Since I was just digging into this, I want to write down what I'velearned before I forget it. I will give you the punchline right upfront: if you want to do IO that avoids ZFS prefetching as much aspossible, you need to use randomized IO . It also turns out that ZFSprefetching has what I consider some flaws that can cause IO heartburn.

(You should use a fixed seed for your randomization function andwatch ZFS statistics to make sure that you really are avoiding theprefetcher.

For reasons beyond the scope of this entry, on some systems Iwatch SMTP logs in fair detail. One result of this is that everyso often I see a burst of unusual SMTP activity ( for example ). Recently I saw a bunch of SMTP attempts overtwo days that looked like this:

24934#  remote from [165.228.246.237]24934r  EHLO [192.168.2

This is something I was looking into today and I want to write stuffdown before it falls out of my head again. I'll start with the premise:suppose that you want to generate unpredictable random numbers to avoid attempts to predict your actions . Do youneed a cryptographically secure (pseudo) random number generator ,or is your language's normal PRNG good enough?

All (pseudo) random number generators ( and hashes )have some amount of internal state that is manipulated and permutedto generate their sequence of

This is a programming puzzle; unlike a lot of what I write about, Idon't have any answers right now.

I would like to find an approach to creating a sequence of IO thatdefeats superintelligent prefetching . Asfar as I can tell the prefetching I'm dealing with recognizes forwardand backwards read IO both straight and with a stride; further, itattempts to find up to N separate streams of prefetchable IO (where Nis probably at least eight). The first puzzle is simple: what