Our current ugly hacks to Dovecot to help mitigate our IMAP problems
Back in the comments of this entry fromthe end of December, I said that we weren't willing to take on thevarious burdens of changing our local Dovecot to add some loggingof things like the mailboxes that people's clients were accessing.In yesterday's entry I mentioned that weactually had hacked up our Dovecot to do exactly that. You mightwonder what happened between December and now to cause us to changeour minds. The short version is that from our perspective, thingson our IMAP server
Some things about Dovecot, its index files, and the IMAP LIST command
We have a backwards compatibility issue with our IMAP server, where people's IMAP roots are $HOME , their home directory,and then clients ask the IMAP server to search all through the IMAPnamespace; this causes various bad things to happen, including running out of inodes . The reason we ranout of inodes is that Dovecot maintains some index files for every mailbox it looksat.
We have Dovecot store its index files on our IMAP server's localdisk, in /var/local/dovecot/<
Why seeing what current attributes a Python object has is hard
Back when I wrote some notes on __slots__ and class hierarchies , I said in passing that there was no simple wayto see what attributes an object currently has (I was sort of talkingabout objects that use __slots__ , but it's actually more general).Today, for reasons beyond the scope of this entry, I feel liketalking about why things work out this way.
To see where things get tricky, I'll start out by talking aboutwhere they'
You probably don't want to run Firefox Nightly any more
Some people like to run Firefox Nightly for various reasons; youcan like seeing what's coming, or want to help Mozilla out by testingthe bleeding edge, or various other things. I myself have in thepast run a Firefox compiled from the development tree (although atthe moment I'm still using Firefox 56 ).Unfortunately and sadly I must suggest that you not do that anymore, and only run Firefox Nightly if you absolutely have to (forexample to test some bleeding edge web feature
Python and the 'bags of unstructured data' approach
These days I write code in both Go and Python, which sometimes givesme interesting new perspectives on each language as I shift backand forth. I was recently hacking on a Python program to mutate it into what I wanted , and as Idid so what struck me is how Python's dynamic typing and everythingaround it enabled a specific approach that I'll call the 'bag ofdata' approach.
The base code I was starting with parses Linux's /proc/self/mountstatsto get at
Some exciting ZFS features that are in OmniOS CE's (near) future
I recently wrote about how much better ZFS pool recovery is coming , which reported on Pavel Zakharov's Turbocharging ZFS Data Recovery . Inthat, Zakharov said that the first OS to get it would likely be OmniOS CE , although he didn't have atimeline. Since I just did some research on this, let's run downsome exciting ZFS features that are almost certainly in OmniOS CE'snear future, and where they are.
There are two big ZFS features from Delphix that have recently landed in
Why I use Facebook (a story of web development)
The reason I actually use my Facebook account is because of mybike club . Explaining this reason reveals somethingsad about the state of the web and putting together web sites, andwhy Facebook is so attractive. I care about this because in somequarters, it's popular to be dismissive of people and groups whouse Facebook despite the many problems with it, and my view is thatpeople who have this attitude may not understand the good reasonsthat push people to Facebook.
(And in turn this matters because if
Much better ZFS pool recovery is coming (in open source ZFS)
One of the long standing issues in ZFS has been that while it'susually very resilient, it can also be very fragile if the wrong things get damaged. Classically,ZFS has had two modes of operation; either it would repair anydamage or it would completely explode. There was no middle groundof error recovery, and this isn't a great experience; as I wroteonce , panicing the system is not an errorrecovery strategy. In early versions of ZFS there was no recoveryat all (you
Wrestling with metrics to get meaningful, useful ones
I'm currently working on hacking together somethingto show us useful information about the most active NFS filesystemson a client (what I called nfsiotop in yesterday's entry ). Linux has copious per-mount statistics and the program that Istarted from already read them all, so a great deal of what I'vebeen doing has been wrestling with the raw data available to comeup with useful metrics and figure out good ways of displaying them.This is a common experience; I have some version of it
What I think I want out of a hypothetical nfsiotop for Linux
I wish there was a version of Linux's nfsiostat that worked gracefullywhen you have several hundred NFS mounts across multiple NFSfileservers.
(I'm going to have to write one, aren't I.)
Linux exposes a very large array of per-filesystem NFS clientstatistics in /proc/self/mountstats (see here )and there are some programs that digest this data and report it,such as nfsiostat(8) . Nfsiostatgenerally works decently to give you useful information,