Why Python's struct module isn't great for handling binary protocols
On the surface, the struct module looks like the right answerif you need to handle a binary protocol .In practice it is not the right answer, or at least it isn't thecomplete answer, because it has some weaknesses and limitations. Theissues are primarily but not entirely in the area of decoding binarymessages as opposed to creating them ('deserialization' versus'serialization').
Note that I am not blaming the struct module for this. Itsdocumentation is very straightforward about what it's
Why I'm interested in Go
These days, my substantial programming takes place in one of twolanguages. I use Python if I'm dealing with something that doesn't haveto run fast or use minimal memory, and I use C on the occasions whenPython doesn't fit. I like C, but it is a very sparse and unforgivinglanguage when compared to Python and this translates to slower andmore annoying development most of the time; there's a lot of low leveldetails that I have to worry about when I just
Why we built our own ZFS spares handling system
I mentioned recently that we'vewritten our own system to handle ZFS spares. Before I describe it, Iwanted to write up something about why we decided to go to the extrememeasure of discarding all of ZFS's own spare handling and rolling ourown.
First off, note that our environment is unusual. We have a lot of poolsand a relatively complex SAN disk topology with at least three levels,as opposed to the more common environment of only a few pools andessentially undifferentiated disks. I expect
My issues with Go's net package
I would like to like the Go language ,but right now I can't get past my annoyance at its net package . I have several network-related thingsthat I would like to do with Go, and the net package's current state isgetting in the way of all of them.
The fundamental problem is that Go's net package is both incomplete andwhat I'll call sealed. The easiest way for me to explain this is tocontrast it with Python's socket support.
Both
My theory on Unix's one chance to have a standard GUI
In an earlier entry I discussed my view on whyUnix vendors never got together and created a standard GUI, the way theycreated POSIX. It's my view that the Unix world had basically one chanceto create a standard GUI but fumbled it through (vendor) greed, althoughsadly understandable vendor greed.
Let's start with a question: why does Unix have a standard graphicalenvironment? Because it does; for at least two decades now, X Windowshas been the ubiquitous Unix way to do graphics (
The problems with testing for round-trip capable codecs
In theory, testing a Python codec to see if it's round-tripcapable is pretty simple:
def willtrip(cname): c1 = [chr(x) for x in range(0, 256)] try: uc = [x.decode(cname) for x in c1] c2 = [x.encode(cname) for x in uc] except UnicodeError: return False return c1 == c2
(Since it's been a while,
Round-trip capable character encodings in Python
Suppose (mostly hypothetically) that you have some bytes in an unknownencoding (or possibly in no encoding because they represent somebinary data), but you need to pass them through some routine that onlyaccepts Unicode strings. What you need is a character encoding that canround-trip arbitrary byte values through Unicode, one that can uniquelyrepresent and reproduce all byte values from 0 to 255.
(This is not true of all character encodings. The 'ascii' codec isonly defined on some bytes,
A module that I want in the standard library: Parsing
Given what I've written before , it willprobably not surprise you that I have a little list of modules thatI think should be in the standard library. The first one of them isa decent parsing or parser generator module.
There are two reasons in combination for this. First off, various sortsof parsing is an important part of the work that a decent number ofprograms do. Full bore language parsers are somewhat uncommon, but (atleast in my work) it's reasonably common to be
The cache eviction death spiral
Here's a cache behavior I've touched on in passing several times butI feel like covering explicitly now. It's what I've taken to call thecache eviction death spiral, where cache evictions start you down a pathto grinding death.
Cache evictions of useful data have an obvious problem; they slow downthe program when you have to pull the needed data back into cache. Butin a contended cache under pressure from multiple sources, they have asecond effect as well; because your program
Read IO is generally synchronous (unlike write IO)
One of the important general difference between read IO and write IO isthat read IO is generally synchronous (in its normal form), and not justbecause the operating system interfaces are usually synchronous. Read IOis synchronous in general because the program is almost always going toimmediately look at and do something with the data that it's read, andit can't go on until it's done this. It's a relatively rare program thatdoesn't look at the data but instead either discards