Back when I wrote about using tc to limit the outgoing bandwidth of a web server , I expressed a wish:

For our purposes it would be nice to do something sophisticated to aggregate all HTTPS requests from a single IP address together in a single 'flow' for tc-sfq(8) . In theory this is possible with tc-flow(8) but in practice I can't figure out a command line that works right (despite consulting eg tc-sfb(8) 's example).

The good news is that I managed to figure out a syntax that tc would accept. The bad news is that I don't know if it does what I want, because I'm not sure how you see what connections are assigned to what flows.

I'll put forward two variations of tc-flow(8) commands. To start with, I'll set the stage. We start with our bandwidth limited class that will have a tc-sfq(8) qdisc below it, straight from the first entry :

tc class add dev eno1 parent 1: classid 1:10 htb rate 400mbit ceil 500mbit prio 10

Our first option is to do exactly what I said above, making the flow per-IP instead of per four-tuple, and attach this flow classification to our bandwidth limited qdisc.

tc filter add dev eno1 protocol ip parent 1:10 handle 20 flow hash keys src,dst,proto,proto-src divisor 4096

(The ' handle <something> ' is apparently critical although I don't know why. I got this wisdom from here after extensive online searches.)

This theoretically makes flows be based on the source IP, destination IP, source port, and the protocol. In our case, for a HTTPS web server with us dealing with outgoing traffic, the source, protocol, and source port are all the same, so this only varies the flow by the destination IP. This is what we want to aggregate all connections from the same IP. Some sources will tell you to use ' perturb N ' on this. We don't want to do this because we want flows to stay sticky and not get re-sorted every so often, and we want to set a very large divisor so that we maintain a lot of distinct flows to go with our (initially) large number of connections .

(The tc-sfb(8) manual page has an example like this, basing flows purely on the destination IP, but it's attached to the root qdisc.)

The other option is to be even more aggressive and theoretically group flows by the /24 of the destination IP address, and nothing else (since we're already theoretically only dealing with HTTPS responses sent by our web server, with the protocol, source IP, and source port all constant). This is done with the other sort of flow filter, a 'map' filter instead of a 'hash':

tc filter add dev eno1 protocol ip parent 1:10 handle 20 flow map key dst rshift 8 divisor 4096

If I'm understanding tc-flow(8) correctly, this takes the destination IP and right shifts it 8 bits (then divides the result by 4096, once again to try to have as many distinct flows as we could in theory have simultaneous connections). Dropping the right octet of a full IP address effectively reduces it to its /24. I think we could also use ' dst and 0xffffff00 ' to get basically the same effect, but maybe that would get more flow collisions when the dust settled.

I believe that we also need to tell our tc-sfq(8) queue how many flows it's supposed to have, matched with the number we used above:

tc qdisc add dev eno1 parent 1:10 handle 10: sfq flows 4096 divisor 4096

Unfortunately, I don't know if all of this works because I don't know of any way to see what flow a given connection is classified into (either before or after tc-sfq(8) gets at it). It could be that my tc-flow(8) usage isn't actually assigning flows the way I want (especially in the second case). It could also be that I need to put my flow filter somewhere else in the tc class hierarchy, or maybe somehow attach it directly to the tc-sfq(8) qdisc.

(Given the example in tc-sfb(8) , where the filter is attached directly to the sfb qdisc, possibly I need to attach the flow filters to the sfq qdisc, with 'parent 10:' instead of 'parent 1:10'. This appears to be accepted by tc , although who knows if it's working.)

PS: Based on what ' iftop ' is telling me, I'm relatively sure that my flow filter isn't actually working the way I want (with a flow filter either in this version or directly attached to the sfq qdisc). But it's hard to be sure.