What 32-bit x86 Linux's odd 896 MB kernel memory boundary is about
Back in my entry on how the Linux kernel divides up your RAM I described the somewhat odd split the 32-bit x86Linux kernel uses. In 32-bit x86 kernels, the Normal zone is onlymemory up to 896 MB and the HighMem zone is everything above it.The reasons for this are rooted in history and the 32-bit (kernel)memory map.
In the beginning, Linux only ran on (32-bit) x86 machines
My dislike for what I call 'perverse Test Driven Development'
There is a particular style of TDD that I will call 'perverse TDD'for lack of a better name. In perverse TDD, you are supposed to veryliterally write the most minimal code possible to pass a new test,even if the code is completely useless and artificial. The ostensiblejustification and excuse for this is that this makes sure you have testsfor all of your code (taken from this site on TDD Django development , because reading itswriteup on this is what pushed me over the edge today).
The kernel memory addressing problem
One of the engineering issues in writing an operating system kernelis how your kernel gets access to physical memory. This requires someexplanation, since on the surface you might think that this is easy;after all, the kernel runs with full access to the machine so how couldit have problems accessing memory?
The simple answer is that today's kernels almost always run with virtualmemory, not just for user processes but for themselves as well. Althoughthey run with full privileges, kernels still have a virtual addressspace
Unicode's two new problems
If you want to convert code from dealing with more or less uninterpretedstrings of bytes into dealing properly with character encodings, ie fromusing raw bytes into using Unicode, you will have two new fundamentalproblems. (You will also have a number of practical problems likesorting, but these can be addressed with suitable Unicode librariesand perhaps a certain amount of handwaving.)
These two problems are what to do with invalid input when you decodefrom byte strings into Unicode and what to do with Unicode code pointsthat can'
A sleazy trick to capture debugging output from an initramfs
Suppose, not entirely hypothetically ,that something in your system's initramfs is failing or that you justwant to capture some debugging output or state information in general.The traditional way to do this when console output isn't good enough isto just dump the output into a file and read the file later, but thishas a problem in the initramfs world; the file you write out will bein the initramfs, which means that it will quietly disappear when bootprocess is finished and the initramfs goes away.
The history of booting Linux with software RAID
One of the broad developments in the Linux kernel's boot process overthe past N years has been a steady move from having the kernel do thingsinside itself to having them done in user level code (which is typicallyrun from an initramfs). The handling of software RAID arrays is noexception to this.
In the beginning, activating software RAID arrays at boot time washandled inside the kernel. At boot time the kernel (specifically thesoftware RAID code) scanned all disk partitions of type fd ('Linux
Ubuntu 12.04 can't reliably boot with software RAID (and why)
Recently one of my co-workers discovered, diagnosed, andworked around a significant issue with software RAID on Ubuntu12.04. I'm writing it up here partly to get it all straight inmy head and partly so we can help out anyone else with the sameproblem. The quick summary of the situation comes from my tweet :
Ubuntu 12.04 will not reliably boot a system with software RAIDarrays due to races in the initramfs scripts.
(As you might guess, I
The temptation of selective sender address verification
You could say that we have at least three mail problems. We have arequirement to accept all email by default, wehave people who either forward all their mail toplaces with more strict spam policies or have autoreplies, and suchpeople sometimes get email from known domains that can't be sent email( sometimes in annoying ways ). The result is a moderateamount of email camped out on our system that I know will never bedelivered, and a sender address verification temptation.
There all sorts of reasons why
Unicode code points and abstract characters
In yesterday's entry I mentioned inpassing that Unicode code points were both more and less than abstractcharacters. Since this is the kind of statement that might raisepeople's eyebrows, I figure that I should explain (and justify) it.
Unicode code points are more than just abstract charactersbecause of the presence of combining characters . Combining charactersturn Unicode code points into a system for assembling an abstractcharacter from components; you have the base character and then variousaccent marks and suchlike added on to it.
Strings in Python 2 and Python 3
This started life as a reply to a comment on my entry about my issueswith Unicode in Python 3 but grew, so I'm makingit into an entry of its own. A commentator wrote:
If you want to read a sequence of bytes -- from, say, a file -- youcan do that in Python 3. You just have to explicitly ask for it, andthe datatype you get back will not be str. It shouldn't be! A str ismeant to represent an