The importance of understanding Python's implementation
Joel Spolsky likes to talk about leaky abstractions ,how the lower level details underneath abstractions inevitably leakthrough and affect your work with those abstractions. High levellanguages like Python are no exception, because how their abstractionsare implemented has significant effects on how fast different sorts ofcode will run.
For instance, let's take lists and tuples, both of which CPythonimplements as arrays of objects. This has a number of consequences:
- indexing a list element is cheap, and all list elements are equallycheap to access.
How doing relative name DNS lookups can shoot you in the foot
DNS-based host name lookups can be what I'll call 'relative', whichlook for the host name inside some domain, or 'absolute', whichassumes that the host name is fully qualified and starts right fromthe root DNS. (For clarity, absolute names are often written with atrailing '.'; this can often be used to make resolver libraries treatthem as absolute.)
Once upon a time, we had an interesting mail explosion. The campuswide mail servers started sending our
Using Python introspection for semi-evil
One of the things I write in Python is network daemons. Because it worksso nicely, network daemons usually take input as text lines that startwith a command word and have whitespace separated arguments. There'sa certain amount of eye-rolling tedium in writing code to check thatthe command word is valid and has the right number of arguments. WhenI wrote a program that does a lot of this, the tedium finally got tome and I sort of snapped and automated all of the validation throughPython'
Fun with DNS: a semi-obscure problem
I just helped a friend troubleshoot a DNS problem he was having on oneof his home Linux machines. The specific problem was that Firefox wastaking a long time to start connecting to web sites; he believed itwas DNS problems.
I ran down some standard guesses:
- maybe one or more nameservers in
/etc/resolv.confweren'tresponding, and Firefox was having to wait for lookups against eachto time out. But straight lookups withdigweren't delayed, so itwasn't that.
Examining Python's string concatenation optimization
In the comments on the other day's' minimizing object churn ' entry, it wasnoted that Python string concatenation had been optimized some in therecent 2.4 release of the CPython interpreter. The 2.4 release notessay (from here ):
- String concatenations in statements of the form
s = s + "abc"ands += "abc"are now performed more efficiently in certaincircumstances.
But what is 'more efficiently' and what are the 'certaincircumstances'? What does it
Weekly spam summary on November 5th, 2005
This week we received 12,872 email messages from 229 different IPaddresses. Our SMTP server handled 22,584 sessions from 1,544different IP addresses, which is significantly down from last week .
To go with it, overall connections are down a lot from last week : we only saw 93,950 connections from at least 31,000different IP addresses. I believe this is the lowest connection rateI've seen since
Minimizing object churn to optimize Python code
One of the interesting things about writing code in high level garbagecollected languages is how your optimization concerns can change. Oneoptimization in all GC'd languages, Python included, is minimizing thecreation of new objects and maximizing the reuse of old ones; thefewer new objects you create, the fewer cycles garbage collectioneats, the less memory gets fragmented, and you often even use lessmemory overall.
Minimizing object churn is behind several well-known Python(anti-)patterns, such as 'avoid repeated string
Modifying Linux kernel configuration options the easy way
Modifying a Linux kernel configuration option can be an annoyingexperience, as you poke around the menus of ' make menuconfig ' tryingto find where the configuration option you want to change is hiding.Fortunately there's a less well known but much easier way.
The kernel build process stores configuration settings in the file .config in the root of the kernel source tree. However, it generatesother things from the file (and some options are interdependent), so youcan't just edit the file and go;
Fun with upgrading our backup server
We've been having odd problems with backups for more than a weeknow. Problems with backups are often frustratingly hard to debug,since generally I only get one run a day and it really should work.
Our Amanda backup server still ran Red Hat7.3 for various reasons (including lack of disk space, which is a longstory). I had been planning to upgrade it to Fedora Core 4 just beforethe backup problems hit, then they pushed that back, then we workedout that they'
How well do some Atom feed fetchers do conditional GETs?
' Conditional GET 'is the HTTP technique used to save bandwidth by not re-fetchingunchanged pages. Using conditional GET is especially important forthings that fetch syndication feeds (RSS or Atom), because peopleusually check feeds much more often than they revisit web pages.( This is another good reference for syndication feed reader authors.)
WanderingThoughts has a lot of syndication feeds and the main ones are quite big. Recently, partly prompted by issues with MSNbot , I decided to take alook at what was fetching my