What ZFS gang blocks are and why they exist

If you read up on ZFS internals, sooner or later you will run acrossreferences to 'gang blocks'. For instance, they came up when Italked about what's in a DVA , whereDVAs have a flag to say that they point to a gang block instead ofa regular block. Gang blocks are vaguely described as being a way of fragmenting a large logical block into a bunch ofseparate sub-blocks.

A more on-point description can be found in the (draft) ZFS on-


Confirming the behavior of file block sizes in ZFS

ZFS filesystems have a property called their recordsize , whichis usually described as something like the following (from here ):

All files are stored either as a single block of varying sizes (up tothe recordsize) or using multiple recordsize blocks.

A while back I wrote about using zdb to peer into how ZFS storesfiles on disk , where I looked into how ZFSstored a 160 Kb file and specifically if it really did use two 128Kb blocks to hold it, instead of a 12


The goals and problems of our Dovecot IMAP configuration migration

We have a long standing backwards compatibility issue with ourIMAP server , which is that we have itconfigured so that the root of the IMAP mail folder storage ispeople's $HOME . Originally this led to Dovecot session hangs , but now it's led to running out ofinodes on the Dovecot server machine andgeneral NFS load as people's Dovecot sessions rummage all throughtheir home directories on our fileservers . Today I'm going to talk aboutour ideal IMAP configuration, the problems of trying to migrate


A brief review of the Dell XPS 13 as a Fedora laptop

For years, my work laptop was a series of old second or third handThinkpads, I believe first a T40 and then a T61. These were relativelybulky and heavy, with short battery lifetimes (about two hours fromfull charge) and only a 1024x768 resolution screen, and their CPUswere both so old that they were 32-bit only machines. It's beenclear for a while that 32-bit machines are on their last legs


Some notes on relative imports and vendor/ in Go

For reasons beyond the scope of this entry, I've become interestedin ways to work with multiple packages in Go without doing it inthe normal way, with a $GOPATH/src and packages and programs thatcan be go get 'd from Github or any of the other places that the go tool supports. The short version of what I'm interested in isthat I'd like to create self contained program source code treesthat are still modularized into multiple Go packages, instead ofthrowing everything into


Understanding IMAP path prefixes in clients and servers

Suppose you have some IMAP clients and they talk to an IMAP serverwhich stores mailboxes somewhere in the filesystem under people'shome directories (let's call this the IMAP root for a user). Oneof the complications of talking about where people's mailboxes andfolders actually wind up in this environment is that both the clientsand the server get to contribute their two cents, but how theymanifest is different.

(As a disclaimer, I'm probably abusing IMAP related terminologyhere in ways that aren


Is the C runtime and library a legitimate part of the Unix API?

One of the knocks against Go is, to quote from Debugging an evilGo runtime bug (partly via ):

Go also happens to have a (rather insane, in my opinion) policy ofreinventing its own standard library, so it does not use any of thestandard Linux glibc code to call vDSO, but rather rolls its own calls(and syscalls too).

Ordinary non-C languages on Unixes generally implement a great manylow level operations by calling into the standard C library. Thisstarts with things like


Some details of ZFS DVAs and what some of their fields store

One piece of ZFS terminology is DVA and DVAs, which is short forData Virtual Address. For ZFS, a DVA is the equivalent of a blocknumber in other filesystems; it tells ZFS where to find whateverdata we're talking about. DVAs are generally embedded into 'blockpointers', and you can find a big comment laying out the entirestructure of all of this in spa.h .The two fields of a DVA that I'm interested in today are the vdev and the offset .

(


To get much faster, an implementation of Python must do less work

Python, like many other dynamically typed languages, is both flexibleand what I'll call mutable . Python's dynamic typing and the powerit gives you over objects means that apparently simple actions canunpredictably do complex things.

As an example of what I mean by this, consider the following code:

def afunc(dct, strct, thing2):  loc = dct["athing"] + thing2  return loc + strct.attr

It's possible and perhaps even very likely that this Python code


How our IMAP server wound up running out of inodes

On Twitter, I mentioned that we'd run out of inodes on a server , and thena few weeks later I made a comment about an IMAP feature :

I'm coming to really dislike IMAP clients that don't usesubscriptions, even though the consequences for our server are sort ofour own fault.

These two tweets are very closely related, and there is a sad storyhere (since it's sort of our own fault).

In the IMAP protocol, there are two ways to get