You are not logged in.
EDIT: my final script is in the post #16 with a short description of what it does!
Hello dear arch community,
I've been trying to set the following stuff up on my server:
- traffic shaping and policying (shape upload and download traffic) of all traffic that leaves the LAN on my arch box
- give SSH, web interfaces of programs like sabnzbd and WLAN users more priority
Some information about my archlinux box
- it is connected to the internet with a router inbetween
- it acts as wireless access point (that already works)
- it stores lots of files, that I want to access fast via LAN. but I don't want it to suck up all the traffic when it runs a download or when I access files on the go over the internet
- it has a vpn, that I only want to use over the LAN
I have tried out various tools (for example trickle - wrote my own archwiki article there), but all have failed me so far
After lots of lots of reading, I decided to try iptables.
I've used this guideas a base, figured out that it doesn't work that way (you can't make sub-classes of classes?), modified my approach, but I still can't get it to work.
Here's my script, that should set up all the iptables and tc stuff:
# config
device="br0"
lan_ip="192.168.2.131"
inet_burst=6k
inet_up=500kbit
inet_down=500kbit
port_sabnzbd=28093
port_pyload=31431
port_ssh=12934
# Classes:
#
# 1: (qdisc) lan(11)
# |
# |----|
# | | (inet down, inet up)
# | |
# | |
# | 1:3 inet_up(13) (Webinterfaces, SSH, WLAN, all other traffic)
# |
# |
# 1:2 inet_down(12) (Webinterfaces, SSH, WLAN, all other traffic)
#
echo "setting up classes..."
tc qdisc add dev ${device} root handle 1: htb default 22
echo "1:2..." && tc class add dev ${device} parent 1: classid 1:2 htb rate ${inet_down} ceil ${inet_down} burst ${inet_burst}
echo "1:3..." && tc class add dev ${device} parent 1: classid 1:3 htb rate ${inet_up} ceil ${inet_up} burst ${inet_burst}
# put marks from IPTABLES to the right classes
echo "adding some iptables->classes glue with tc..."
tc filter add dev ${device} protocol ip parent 1: prio 3 handle 11 fw classid 1:
tc filter add dev ${device} protocol ip parent 1: prio 2 handle 12 fw classid 1:2
tc filter add dev ${device} protocol ip parent 1: prio 2 handle 13 fw classid 1:3
# high priorty inet mark
tc filter add dev ${device} protocol ip parent 1: prio 1 handle 22 fw classid 1:2
tc filter add dev ${device} protocol ip parent 1: prio 1 handle 23 fw classid 1:3
# add iptables rules
echo "adding iptables rules..."
# lan
iptables -I PREROUTING -p all -i ${device} -s "192.168.2.0/24" -d "192.168.2.0/24" -t mangle -j MARK --set-mark 11
# web interfaces
iptables -I PREROUTING -p TCP -i ${device} -s ${lan_ip} --sport ${port_sabnzbd} -t mangle -j MARK --set-mark 12 -v
iptables -I PREROUTING -p TCP -i ${device} -s ${lan_ip} --sport ${port_pyload} -t mangle -j MARK --set-mark 12 -v
iptables -I PREROUTING -p TCP -i ${device} -d ${lan_ip} --dport ${port_sabnzbd} -t mangle -j MARK --set-mark 31 -v
iptables -I PREROUTING -p TCP -i ${device} -d ${lan_ip} --dport ${port_pyload} -t mangle -j MARK --set-mark 31 -v
# ssh
iptables -I PREROUTING -p TCP -i ${device} -s ${lan_ip} --sport ${port_ssh} -t mangle -j MARK --set-mark 22 -v
iptables -I PREROUTING -p TCP -i ${device} -d ${lan_ip} --dport ${port_ssh} -t mangle -j MARK --set-mark 23 -v
# wlan + downloads + everything else
iptables -I PREROUTING -p all -i ${device} -s ${lan_ip} -t mangle -j MARK --set-mark 12 -v
iptables -I PREROUTING -p all -i ${device} -d ${lan_ip} -t mangle -j MARK --set-mark 13 -v
It doesn't give any errors anymore, but when I try to wget the newest ubuntu release for example, I still have the full speed (it should be in the 1:2 class and therefore limited!)
Oh and I am not even sure if $device should be eth0 or br0 (since I also use br0 for dhcpcd), but I've tried both over and over -- nothing works
So if anyone could help me with that, it would be greatly appreciated! You may also suggest me another tool if you know one that fits (although doing the hard way and learning what's wrong here would be preferred and more arch-way-ish of course )
Thanks a lot for reading!
Here's some more console output:
% iptables-save
% ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 6000 qdisc pfifo_fast master br0 state UP qlen 1000
link/ether 00:01:2e:27:59:e0 brd ff:ff:ff:ff:ff:ff
inet6 fe80::201:2eff:fe27:59e0/64 scope link
valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP qlen 1000
link/ether 00:24:23:08:dd:a0 brd ff:ff:ff:ff:ff:ff
inet6 fe80::224:23ff:fe08:dda0/64 scope link
valid_lft forever preferred_lft forever
4: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 6000 qdisc pfifo_fast state UNKNOWN qlen 100
link/none
inet 10.8.0.1 peer 10.8.0.2/32 scope global tun0
14: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP
link/ether 00:01:2e:27:59:e0 brd ff:ff:ff:ff:ff:ff
inet 192.168.2.131/24 brd 192.168.2.255 scope global br0
inet6 fe80::201:2eff:fe27:59e0/64 scope link
valid_lft forever preferred_lft forever
% ip route show table all
default via 192.168.2.1 dev br0 metric 214
10.8.0.0/24 via 10.8.0.2 dev tun0
10.8.0.2 dev tun0 proto kernel scope link src 10.8.0.1
192.168.2.0/24 dev br0 proto kernel scope link src 192.168.2.131 metric 214
local 10.8.0.1 dev tun0 table local proto kernel scope host src 10.8.0.1
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1
broadcast 192.168.2.0 dev br0 table local proto kernel scope link src 192.168.2.131
local 192.168.2.131 dev br0 table local proto kernel scope host src 192.168.2.131
broadcast 192.168.2.255 dev br0 table local proto kernel scope link src 192.168.2.131
local ::1 dev lo proto kernel metric 256
unreachable fe80::/64 dev lo proto kernel metric 256 error -101
fe80::/64 dev eth0 proto kernel metric 256 mtu 6000
fe80::/64 dev wlan0 proto kernel metric 256
fe80::/64 dev br0 proto kernel metric 256
unreachable default dev lo table unspec proto kernel metric 4294967295 error -101 hoplimit 255
local ::1 via :: dev lo table local proto none metric 0 rtt 10ms rttvar 10ms cwnd 10
local fe80::201:2eff:fe27:59e0 via :: dev lo table local proto none metric 0
local fe80::201:2eff:fe27:59e0 via :: dev lo table local proto none metric 0
local fe80::224:23ff:fe08:dda0 via :: dev lo table local proto none metric 0
unreachable ff00::/8 dev lo table local metric 256 error -101
ff00::/8 dev eth0 table local metric 256 mtu 6000
ff00::/8 dev wlan0 table local metric 256
ff00::/8 dev br0 table local metric 256
unreachable default dev lo table unspec proto kernel metric 4294967295 error -101 hoplimit 255
EDIT: I forgot to add this:
% iptables -L -v
Chain INPUT (policy ACCEPT 4657 packets, 2120K bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 7341 packets, 4496K bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 3550 packets, 552K bytes)
pkts bytes target prot opt in out source destination
Shouldn't there be some other rules listed or something?
Last edited by robotanarchy (2012-11-29 10:31:35)
Offline
EDIT: I forgot to add this:
% iptables -L -v Chain INPUT (policy ACCEPT 4657 packets, 2120K bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 7341 packets, 4496K bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 3550 packets, 552K bytes) pkts bytes target prot opt in out source destination
Shouldn't there be some other rules listed or something?
Yes, but they should go in the mangle table:
iptables -nvL -t mangle
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
Thanks, fukawi2!
Your post helped me a lot, because with
watch -n 1 iptables -nvL -t mangle
it became possible to find out what's going on. And that really kept me going
There was a lot wrong in my script, but I finally got the basic stuff to work (seperate speed for LAN and INET) and I think I understand it enough now (manpages tc and tc-htb help a lot) to add what I need
If anyone faces a similar problem, this (working!) script should get you started:
http://mldonkey.sourceforge.net/TrafficShaping
EDIT: not marking this thread as solved yet, as I might still have some questions
Last edited by robotanarchy (2012-11-25 21:58:22)
Offline
No problems; I'm not that familiar with tc etc, but your iptables issue stood out
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
Here's my traffic-shaping setup.
Offline
Currently I have the LAN up/down speed unlimited and the upload inet speed limited.
My problem: Limit inet download speed.
I have a big issue with this line:
iptables -I POSTROUTING -d ${lan_ip} -t mangle -j MARK --set-mark 13 # 0xd
When used like that, with POSTROUTING, the rule does not get any hits. It does get all the hits however, when I use PREROUTING or INPUT, but then "tc" ignores the marks.
I don't quite understand the iptables chain with prerouting, input, output, postrouting and where traffic shaper (tc) kicks in.
There are even some scripts out there that use prerouting (like the one I originally based mine on), but I can't get them to work.
Do I need to tell traffic shaper to look at the prerouting traffic somehow? Is this even possible?
Here's my full script:
# config
device="br0"
lan_ip="192.168.2.131"
inet_burst="6k"
inet_up="100000bit"
inet_down="100000bit"
# reset everything
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -F
iptables -t nat -F
iptables -t mangle -F
tc qdisc del dev $device root 2> /dev/null
iptables -F POSTROUTING -t mangle
# Classes:
#
# 1: (qdisc)
# |
# |----|----|
# | | | (inet down, inet up)
# | | |
# | | |
# | | 1:13 inet_down(13) (Webinterfaces, SSH, WLAN, all other traffic)
# | |
# | |
# | 1:12 inet_up(12) (Webinterfaces, SSH, WLAN, all other traffic)
# |
# |
# 1:11 lan_traffic (11)
echo "setting up tc qdisc and classes..."
tc qdisc add dev ${device} root handle 1: htb default 13
echo "1:11..." && tc class add dev ${device} parent 1: classid 11 htb rate 1tbps burst 12k
echo "1:12..." && tc class add dev ${device} parent 1: classid 12 htb rate ${inet_up} burst ${inet_burst}
echo "1:13..." && tc class add dev ${device} parent 1: classid 13 htb rate ${inet_down} burst ${inet_burst}
# put marks from IPTABLES to the right classes
echo "adding tc filters to map iptables to classes..."
tc filter add dev ${device} parent 1:0 prio 3 protocol ip handle 11 fw flowid 1:11
tc filter add dev ${device} parent 1:0 prio 2 protocol ip handle 12 fw flowid 1:12
tc filter add dev ${device} parent 1:0 prio 2 protocol ip handle 13 fw flowid 1:13
# high priorty inet mark
#tc filter add dev ${device} protocol ip prio 1 handle 22 fw classid 12
#tc filter add dev ${device} protocol ip prio 1 handle 23 fw classid 13
echo "adding iptables rules..."
# POSTROUTING for upload traffic
echo "lan and vpn..." # needs to be the last rule, so put it on top here
iptables -I POSTROUTING -s "192.168.2.0/24" -d "192.168.2.0/24" -t mangle -j MARK --set-mark 11 # 0xb
echo "wlan, downloads, everything else..."
iptables -I POSTROUTING -s ${lan_ip} -t mangle -j MARK --set-mark 12 # 0xc
# PREROUTING for download traffic
#
# HELP
#
# only works with PREROUTING or INPUT but then tc doesn't seem to use the marks!
#
#
#
iptables -I PREROUTING -d ${lan_ip} -t mangle -j MARK --set-mark 13 # 0xd
# ssh, webinterfaces: todo, but that's not so hard
And thanks brebs, for posting your script!
Last edited by robotanarchy (2012-11-25 22:52:06)
Offline
Limit inet download speed.
Who says that's a good idea?
AFAICT, that's pointless. Just shape the uploading, and let the downloading be done as normal (we cannot control the downloading).
Edit: Added link.
Last edited by brebs (2012-11-25 23:00:36)
Offline
robotanarchy wrote:Limit inet download speed.
Who says that's a good idea?
AFAICT, that's pointless. Just shape the uploading, and let the downloading be done as normal.
From what I have read, you can't directly control download speed, but at least you can set up a policy that ACKs get delayed/dropped and hopefully the host won't send that much data then.
I would like to have this, because the server is in a LAN environment and its downloads shouldn't block others from streaming youtube for example.
EDIT: If you know a better way, let me know. But I can not modify the hosts I download from to send less data
Last edited by robotanarchy (2012-11-25 23:00:49)
Offline
ACKs get delayed/dropped
That would just cause *more* TCP re-synchronization/handshaking traffic. Counter-productive
Offline
well it works in download managers I use what would you suggest then?
Offline
Download managers just add a crappy artificial speed limit. They aren't capable of using all the appropriately-allocated available bandwidth, as it dynamically ebbs and flows. This is why iptables shaping/sharing is better.
I've already said what to do. Concentrate on shaping uploads only.
Are you afraid of the LAN being saturated? You say "its downloads" - do you mean the server's downloads? I'm struggling to understand your logic.
Offline
Oh sorry, what I meant with "the server's downloads" is when someone uploads something to the server.
Or when I want to download something with the server, in that case I mostly can't limit the speed on the other side.
Basic example:
I run
wget http://big/file
on the server to download something, that's where I would like to limit the speed
Offline
The speed is limited anyway, because the sender will be waiting for acknowledgement-of-receipt packets from your server.
And once the data has reached your server, it's gone beyond the appropriate time to drop it.
Offline
....set up a policy that ACKs get delayed/dropped
That IS controlling/policing the upload, not the download. There is no way to directly control how fast the other end sends data, and any indirect methods are inaccurate.
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
The speed is limited anyway, because the sender will be waiting for acknowledgement-of-receipt packets from your server.
And once the data has reached your server, it's gone beyond the appropriate time to drop it.
As far as I know, it works like this:
Drop ACK packages that supply more packages than I want per second -> The server (probably) sends less data per second -> Less packages/data/traffic per second for real
robotanarchy wrote:....set up a policy that ACKs get delayed/dropped
That IS controlling/policing the upload, not the download. There is no way to directly control how fast the other end sends data, and any indirect methods are inaccurate.
Well, I prefer inaccurate indirect control over no control
I'm currently playing with "ingress", let's see if that gets me anywhere.
Last edited by robotanarchy (2012-11-27 22:30:54)
Offline
Yay, it works now like I want it to
Here's my script if anyone wants something similar. I have read through mastershaper scripts, that helped a lot
again, here's the purpose:
Shape/Policy all traffic to a maximum down-/ and upload speed, except for LAN traffic (which is always fullspeed). Also SSH and (in this case pyload and sabnzbd)-webinterfaces are prioritized, customize to your needs
device="br0"
lan_ip="192.168.2.131"
lan_mask="192.168.2.0/24"
inet_burst="10k"
inet_up="1000kbps"
inet_down="3000kbps"
port_sabnzbd=3233
port_pyload=5334
port_ssh=1123
echo "reseting everything traffic shaping related..."
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -F
iptables -t nat -F
iptables -t mangle -F
tc qdisc del dev $device root 2> /dev/null
tc qdisc del dev $device ingress 2> /dev/null
iptables -F POSTROUTING -t mangle
echo "setting up tc qdiscs..."
echo " ingress..." && tc qdisc add dev ${device} handle ffff: ingress
echo " htb..." && tc qdisc add dev ${device} root handle 1: htb default 13
echo "seting up classes..."
echo " 1:11..." && tc class add dev ${device} parent 1: classid 11 htb rate 1tbps burst 12k
echo " 1:22..." && tc class add dev ${device} parent 1: classid 12 htb rate ${inet_up} burst ${inet_burst}
echo "adding tc filters to map marks from iptable rules to classes..."
echo " filter 11(0x0b)->1:11 (prio 3)..." && tc filter add dev ${device} parent 1:0 prio 3 protocol ip handle 11 fw flowid 1:11
echo " filter 12(0x0c)->1:12 (prio 2)..." && tc filter add dev ${device} parent 1:0 prio 2 protocol ip handle 12 fw flowid 1:12
echo " filter 22(0x16)->1:12 (prio 1)..." && tc filter add dev ${device} parent 1:0 protocol ip prio 1 handle 22 fw flowid 1:12
echo "adding iptables rules..."
echo " lan and vpn..." && iptables -I POSTROUTING -s ${lan_mask} -d ${lan_mask} -t mangle -j MARK --set-mark 11 # 0xb
echo " ssh..." && iptables -I POSTROUTING -p TCP -s ${lan_ip} --sport ${port_ssh} -t mangle -j MARK --set-mark 22 # 0x16
echo " sabnzbd..." && iptables -I POSTROUTING -p TCP -s ${lan_ip} --sport ${port_sabnzbd} -t mangle -j MARK --set-mark 22 # 0x16
echo " pyload..." && iptables -I POSTROUTING -p TCP -s ${lan_ip} --sport ${port_pyload} -t mangle -j MARK --set-mark 22 # 0x16
echo " wlan, downloads, everything else..." && iptables -I POSTROUTING -s ${lan_ip} -t mangle -j MARK --set-mark 12 # 0xc
echo "adding download limit filter..."
echo " lan..." && tc filter add dev ${device} parent ffff: protocol ip prio 1 u32 match ip src ${lan_mask} flowid :1
echo " other..." && tc filter add dev ${device} parent ffff: protocol ip prio 50 u32 match ip src 0.0.0.0/0 police rate ${inet_down} burst ${inet_burst} drop flowid :1
EDIT: modified the "lan..." line according to this thread: http://forums.opensuse.org/english/get- … luded.html because it didn't really do what I wanted before
Last edited by robotanarchy (2012-12-11 23:40:33)
Offline
What benefit do you see from those two "police rate" lines? AFAICT, they only make sense if there's some other traffic on your LAN (e.g. PC-to-PC, not going through the server) which you don't want to congest.
Which is a different reason to the ones you were giving
Offline
What benefit do you see from those two "police rate" lines? AFAICT, they only make sense if there's some other traffic on your LAN (e.g. PC-to-PC, not going through the server) which you don't want to congest.
Which is a different reason to the ones you were giving
Oh that's the point where you didn't understand me? Because that was exactly the point. There are lots of PCs on the LAN, which directly get stuff from the internet, not through the pc where the script runs on. Like this:
router ----> archbox_with_that_script ---> only a few mobiles/pcs with wlan
'------> many more pcs
I hope it's clear now and you're right, I've read my posts again and I didn't make that clear earlier, sorry for that
Offline