Microsoft's Bingbot crawler is on a relative rampage here

For some time, people in various places have been reporting thatMicrosoft Bing's web crawler is hammering them; for example, Discourse has throttled Bingbot ( via ).It turns out that Wandering Thoughts is no exception, soI thought I'd generate some numbers on what I'm seeing.

Over the past 11 days (including today), Bingbot has made 40998requests, amounting to 18% of all requests. In that time it's askedfor only 14


My new 4K HiDPI display really does make a visible difference

As peculiar as it sounds, I had no idea if I'd see a clear visibledifference from moving up to my new 4K HiDPI display before I gotit. In fact I more or less expected it to not be a big change frommy previous monitor (a 24" 16:10 Dell U2412M). I'd read peopleraving about the Retina displays on Mac laptops, but people havedifferent standards for what they consider rave-worthy (and what


What sorts of good email attachments our users get (April 2018 edition)

I've looked at various breakdowns of bad attachment types that getsent to our users, but of course that's not the only reason wecollect all of this data. In fact it's the lesser reason; the greaterone is to know the legitimate types of files our users get inemail . So today I'm going to look at aweek's worth of data from our central mail server , which is logged after allrejecting, filtering, and spam tagging has been applied.

Over


Some notes on Firefox's current media autoplay settings

I am quite violently against videos ever auto-playing in my regularbrowser, under basically any circumstances ever (including oneslike the videos that Twitter uses for those GIFs that people putin their tweets). I hate it with audio, I hate it without audio, Ihate it on the web page I'm currently reading, I hate it on theweb page in another tab. I just hate it.

I've traditionally used some number of extensions to control thisbehavior on prominent offenders like YouTube (


The shifting goals of our custom NFS mount authorization system

We've been doing custom authorization for NFS mounts in our overallenvironment for a very long time. Our most 'recent' system for thisis the NSS-based custom NFS mount authorization scheme that we introduced on our originalSolaris fileservers and now run on our OmniOS-based fileservers ;this system has now been running for on the order of a decade. Inone sense how this system operates has remained the same over thattime (in that it still uses the same basic mechanics); in anothersense,


An implementation difference in NSS netgroups between Linux and Solaris

NSS is the Name Service Switch , or as wenormally know it, /etc/nsswitch.conf . The purpose of NSS is toprovide a flexible way for sysadmins to control how various thingsare looked up, instead of hard-coding it. For flexibility andsimplicity, the traditional libc approach is to use loadable sharedobjects to implement the various lookup methods that nsswitch.conf supports. The core C library itself has no particular knowledge ofthe files or dns nsswitch.conf lookup type; instead that's


What the core practical problems with HiDPI seem to be on Linux

As I've been planning for some time ,I recently got a new high-resolution, high pixel density monitornow that my new home machine can drive one.Specifically I got the 27" Dell P2715Q, which still appears to bethe best overall choice for a 4K HiDPI monitor. As I expected,switching over to a HiDPI display in my unusual fvwm-based Xenvironment has not been a trouble-freeexperience where things just worked. After wrestling with variousissues


ZFS's recordsize as an honest way of keeping checksum overhead down

One of the classical tradeoffs of using checksums to verify theintegrity of something (as ZFS does) is the choice of how large achunk of data to cover with a single checksum. A large chunk sizekeeps the checksum overhead down, but it means that you have toprocess a large amount of data at once in order to verify or createthe checksum. A large size also limits how specific you can be aboutwhat piece of data is damaged, which is important if you want tobe able to recover some


Thinking about why ZFS only does IO in recordsize blocks, even random IO

As I wound up experimentally verifying ,in ZFS all files are stored as a single block of varying size upto the filesystem's recordsize , or using multiple recordsize blocks. As is perhaps less well known, a ZFS logical block is the minimum size of IO to afile, both for reads and especially for writes. Since the default recordsize is 128 Kb, this means that many files of interest arehave recordsize blocks and thus all IO to them is done in 128 Kbunits, even


Spam from Yahoo Groups has quietly disappeared

Over the years I have written several times about what was, at the time, an ongoing seriousand long-term spam problem with email from Yahoo Groups. Not onlywas spam almost all of the Groups email that we got, but it wasalso clear that Yahoo Groups was allowing spammers to create theirown mailing lists. I was coincidentally reminded of this historyrecently, so I wondered how things were today.

One answer is that spam from Yahoo Groups has disappeared. Oh, it'snot completely and utterly