Suppose, not hypothetically, that a single IP address with a single User-Agent has made 557 requests for your blog 's syndication feed in about 22 and a half hours ( most of which were rate-limited and got HTTP 429 replies ). If we generously assume that these requests were distributed evenly over one day (24 hours), what was the average interval between requests (the rate of requests)? The answer is easy enough to work out and it's about two and a half minutes between requests, if they were evenly distributed.
I've been looking at numbers like this lately and I don't feel like working out the math each time, so here is a table of them for my own future use.
| Total requests | Theoretical interval (rate) |
| 6 | Four hours |
| 12 | Two hours |
| 24 | One hour |
| 32 | 45 minutes |
| 48 | 30 minutes |
| 96 | 15 minutes |
| 144 | 10 minutes |
| 288 | 5 minutes |
| 360 | 4 minutes |
| 480 | 3 minutes |
| 720 | 2 minutes |
| 1440 | One minute |
| 2880 | 30 seconds |
| 5760 | 15 seconds |
| 8640 | 10 seconds |
| 17280 | 5 seconds |
| 43200 | 2 seconds |
| 86400 | One second |
(This obviously isn't comprehensive; instead I want it to give me a ballpark idea, and I care more about higher request counts than lower ones. But not too high because I mostly don't deal with really high rates. Every four hours and every 45 minutes are relevant to some ratelimiting I do .)
Yesterday there were about 20,240 requests for the main syndication feed for Wandering Thoughts , which is an aggregate rate of more than one request every five seconds. About 10,570 of those requests weren't blocked in various ways or ratelimited, which is still more than one request every ten seconds (if they were evenly spread out, which they probably weren't).
(There were about 48,000 total requests to Wandering Thoughts , and about 18,980 got successful responses, although almost 2,000 of those successful responses were a single rogue crawler that's now blocked. This is of course nothing compared to what a busy website sees. Yesterday my department's web server saw 491,900 requests, although that seems to have been unusually high. Interested parties can make their own tables for that sort of volume level.)
It's a bit interesting to see this table written out this way. For example, if I thought about it I knew there was a factor of ten difference between one request every ten seconds and one request every second, but it's more concrete when I see the numbers there with the extra zero.