[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

simulating network latency on linux using iproute2



These are some notes I have on how to configure a network interface
to delay output packets to a particular host.  I would say, all in all,
this is much more complicated than seems necessary.  I had thought
iptables could do this, but I've never found any description of this;
instead every relevant reference I could find pointed towards using
"tc" which is part of "iproute2".

Prerequisite: a linux machine with a generic 2.4.26 kernel or
equivalent.  Ie, probably kernel transcript mdw-kern-generic-2.4.26-1.T
(for the time being, that is what I happen to have installed on
some random machines: aardvark, francis, warris, ganesh, & lancashire).
The specific module that must be included is "sch_delay.o".
"sch_prio.o" and "cls_u32.o" are also needed.
Our current umce linux server kernels will NOT work, because
they don't have CONFIG_NET_SCHED turned on.  This doesn't
matter unless people think they might want to use iproute2 with
a umce linux production kernel.

Also needed: a copy of exactly the right version of iproute2.
It turns out this is fairly kernel version specific, at
least to get the necessary functionality for this.
The required functionality only existed in iproute2 for
about 2 months, between these changes:
	2004-06-25  Stephen Hemminger  <shemminger@xxxxxxxx>
		* Add loss parameter to delay
		* Rename delay qdisc to netsim
	2004-04-15  Stephen Hemminger  <shemminger@xxxxxxxx>
		* Add the delay (network simulation scheduler)
This is the version I used:
http://developer.osdl.org/dev/iproute2/download/iproute2-2.6.7-ss040608.tar.gz
this won't build cleanly on umce linux, because the required older berkeley
db library doesn't exist by default.  That means it will blow up building
misc/arpd.c, which isn't needed.  When building, iproute2 must be pointed
at a copy of the kernel headers for the kernel.  You'll probably need
to acquire source for the kernel you're using to do this, and you
might need to configure that kernel source as well.

The result of all this is "tc/tc".  Copy this to /usr/sbin/tc
on the target machine.  tc is basically a way to define a set
of nodes that mostly operate on packets being sent from a network
interface.  Some documentation on this is here:
	http://lartc.org/howto/index.html
however, the stuff on latency is wrong (because they're using
a newer and more advanced feature called netem, which replaces the
delay network scheduler found in 2.4.26.)

Possibly this should all be turned into a transcript with build notes
and such, if other people find they might find this useful.  But not tonight.

This sets up the necessary bits,

tc qdisc add dev eth0 parent 1:3 handle 30: delay 200ms
tc qdisc add dev eth0 parent 1:3 handle 30: delay latency 200ms rate 10mbit
tc filter add dev eth0 protocol ip parent 1:0 prio 3 u32 match ip dst 141.213.229.84 flowid 10:3

At this point,
ping 141.213.229.84
should report 240-250 ms delays, and
tc -s qdisc ls dev eth0
will report all sorts of mysterious data on what was done.

To temporarily turn this off:
tc filter del dev eth0 parent 1:0 prio 3
and on again,
tc filter add dev eth0 protocol ip parent 1:0 prio 3 u32 match ip dst 141.213.229.84 flowiid 10:3

To make it faster,
tc qdisc change dev eth0 parent 1:3 handle 30: delay latency 0.1ms rate 100mbit
the actual delay seems to be about 10 ms, probably due to granularity in
the linux clock logic.

To make it slow again,
tc qdisc change dev eth0 parent 1:3 handle 30: delay latency 200ms rate 10mbit
and here the delay is about 250 ms.

So, this works.  There seems to be a lot of other clever stuff that
can be done with this, that can be mostly described as "clever network tricks".

				-Marcus