I have a large ftp transaction in progress, with an ADSL-connected machine as the client, and my life would be more comfortable if other things (specifically X11 connections, with the same ADSL-connected machine as the X server) got priority over the ftp transaction for using the ADSL modem.
I tried renice 20 [PID of the ftp client], which reported that it had changed the priority of the ftp client from 0 to 19, but this doesn't seem to have speeded up my X applications any, so I'm guessing that nice priority settings only apply to CPU, not to network bandwidth: is this right? If so, how can I make the ftp client be "nice" about network usage?
On 23 Sep 2004, at 14:42, Dan Hatton wrote:
I have a large ftp transaction in progress, with an ADSL-connected machine as the client, and my life would be more comfortable if other things (specifically X11 connections, with the same ADSL-connected machine as the X server) got priority over the ftp transaction for using the ADSL modem.
I tried renice 20 [PID of the ftp client], which reported that it had changed the priority of the ftp client from 0 to 19, but this doesn't seem to have speeded up my X applications any, so I'm guessing that nice priority settings only apply to CPU, not to network bandwidth: is this right? If so, how can I make the ftp client be "nice" about network usage?
If you want to control the bandwidth.. I can recommend 'trickle'
http://monkey.org/~marius/pages/?page=trickle
Trickle is a portable lightweight userspace bandwidth shaper.
I'm not sure if this answers your question, I'm just shooting in the dark here ;)
Craig
On Thu, 23 Sep 2004, Craig wrote:
If you want to control the bandwidth.. I can recommend 'trickle'
I'm not sure if this answers your question, I'm just shooting in the dark here ;)
It's a good-ish approximation. The manpage says that trickle restricts (for example) the ftp client never to use more than some fixed, user-chosen amount of bandwidth. What I'd really like is something more automated, that restricts the ftp client to using some very small bandwidth when other processes are trying to use the network, but lets the ftp client use all the bandwidth it likes when no other process is trying to use the network.
I'm rather concerned, though, by the way Debian's package information claims that trickle needs much a more recent shared C library and GCC support library than I've got, and some other library called libevent, of which my system has never heard. I might give it a whirl, but suspect I'll have compilation difficulties.
Dan Hatton dan.hatton@btinternet.com writes:
I tried renice 20 [PID of the ftp client], which reported that it had changed the priority of the ftp client from 0 to 19, but this doesn't seem to have speeded up my X applications any, so I'm guessing that nice priority settings only apply to CPU, not to network bandwidth: is this right? If so, how can I make the ftp client be "nice" about network usage?
That is indeed right. You want to use traffic shaping.
http://lartc.org/ http://www.greenend.org.uk/rjk/2004/tc.html
On Thu, 23 Sep 2004, Richard Kettlewell wrote:
I tried renice 20 [PID of the ftp client], which reported that it had changed the priority of the ftp client from 0 to 19, but this doesn't seem to have speeded up my X applications any, so I'm guessing that nice priority settings only apply to CPU, not to network bandwidth: is this right? If so, how can I make the ftp client be "nice" about network usage?
That is indeed right. You want to use traffic shaping.
Thanks: the documentation pointers allowed me to work out that I need something along the lines of
tc qdisc add dev ppp0 root sfq
which, if I've understood correctly, is strictly"scheduling" rather than "shaping." Unfortunately, this returns the error message
RTNETLINK answers: Invalid argument
and doesn't seem to do anything. A quick google reveals that many people have the same error message, but I'm not sure about the relevance of any of their scenarios.
I fear that the following, from the tc-sfq manpage, might have something to do with it:
Most often, cable modems and DSL devices do not fall into this category. The same holds for when connected to a switch and trying to send data to a congested segment also connected to the switch.
In this case, the effective queue does not reside within Linux and is therefore not available for scheduling,
for verily am I using an ADSL modem. Any more ideas, please?
On Tue, 28 Sep 2004 20:44:25 +0100 (BST), Dan Hatton dan.hatton@btinternet.com was rumoured to have said:
On Thu, 23 Sep 2004, Richard Kettlewell wrote:
I tried renice 20 [PID of the ftp client], which reported that it had changed the priority of the ftp client from 0 to 19, but this doesn't seem to have speeded up my X applications any, so I'm guessing that nice priority settings only apply to CPU, not to network bandwidth: is this right? If so, how can I make the ftp client be "nice" about network usage?
That is indeed right. You want to use traffic shaping.
Thanks: the documentation pointers allowed me to work out that I need something along the lines of
tc qdisc add dev ppp0 root sfq
which, if I've understood correctly, is strictly"scheduling" rather than "shaping." Unfortunately, this returns the error message
RTNETLINK answers: Invalid argument
and doesn't seem to do anything. A quick google reveals that many people have the same error message, but I'm not sure about the relevance of any of their scenarios.
I fear that the following, from the tc-sfq manpage, might have something to do with it:
Most often, cable modems and DSL devices do not fall into this category. The same holds for when connected to a switch and trying to send data to a congested segment also connected to the switch.
In this case, the effective queue does not reside within Linux and is therefore not available for scheduling,
for verily am I using an ADSL modem. Any more ideas, please?
In short, you need to move the bottleneck somewhere under your control and use a class-based qdisc to define three classes probably: X, ftp and everything else. Each can then be tied to an sfq qdisc for fairness among flows in the same class.
Sometime ago I configured traffic shaping on my linux router to keep things like ssh sessions and IM applications responsive even in the presence of high traffic. Because IMs are usually short and ssh sets TCP_NODELAY on the socket to disable Nagle's algorithm for interactive sessions, the simplest way to do this was to give priority to small packets. You seem to be trying to do something similar, so here's the configuration for the above scenario. It's been a long time since I did this though, so I'm unlikely to be able to offer much more detail than that in the comments ;)
Aim: give priority to packets with length <= 500 bytes Topology: localnet <-> (eth1) linux_router (eth0) <-> adsl_router <-512/256 ADSL-> upstream
------------------------------ htb.sh ------------------------------ #!/bin/sh
#root tc qdisc add dev eth0 root handle 1: htb r2q 1
# common parent so that the 2 children below can borrow from each other. # reduced the rate a little (from the max downstream) since the bottleneck # must be the linux_router<->adsl_router link. ceil must be at least # as big as that of any of the children. tc class add dev eth0 parent 1: classid 1:1 htb rate 245kbit ceil 250kbit
# queue for small packets (0-500), rate ~= downstream/20 + (tweak) tc class add dev eth0 parent 1:1 classid 1:10 htb prio 0 rate 29kbit # queue for everything else, rate is whatever is left tc class add dev eth0 parent 1:1 classid 1:20 htb prio 1 rate 216kbit ceil 250kbit
# and the fair scheduling tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10
# filters; packets are marked by the kernel's netfilter tc filter add dev eth0 parent 1:0 protocol ip handle 3 fw flowid 1:10 tc filter add dev eth0 parent 1:0 protocol ip handle 4 fw flowid 1:20 ------------------------------ htb.sh ------------------------------
-------------------------- iptables rules -------------------------- #!/bin/sh
# for packets routed through iptables -t mangle -A FORWARD -m length --length 0:500 -j MARK --set-mark 0x3 iptables -t mangle -A FORWARD -m length --length 501: -j MARK --set-mark 0x4 # for locally generated packets iptables -t mangle -A OUTPUT -o eth0 -m length --length 0:500 -j MARK --set-mark 0x3 iptables -t mangle -A OUTPUT -o eth0 -m length --length 501: -j MARK --set-mark 0x4 -------------------------- iptables rules --------------------------
There are definitely a lot of knobs to turn here, and I *think* I orignally wrote this for a different topology, with the adsl_router receiving from localnet via the linux_router, but sending directly (don't ask). It still seems to work without any apparent negative impact, but YMM^HWV.
HTH!
rgds, /-sb.
-
Craig -
Dan Hatton -
Richard Kettlewell -
Stelios Bounanos