ArchRouter

ninesharp · 2010-03-09 22:18:57

Hello fellow Archers!

I've decided to make a router out of an old computer I have hanging around, to better support the ~20 people an aging WRT54G supports now. I intend this to be the main thread on this topic, since I'm going to try to document my steps, and possibly even create a web interface so that managing and setting up a router based on arch is a simple task!

This is something I've planned on for a while now, and a large amount of research has gone into it, with respect of what tools to use, etc. That said, I still have some questions, so please do voice your suggestions.

First some background. I have a 10Mbit line to work with, 512kbit up. This services about 20 people in a residential setting, with some moderate torrent traffic, lots of web and email, youtube, and occasionally skype and games. We currently use a WRT54G modded to use the Tomato firmware as the main router, which includes QoS functionality, and wireless access. There is another, more powerful D-link 615 chained off of it, the only function of which is wireless access. I would use the D-Link as the main router, but it lacks true QoS (with otherwise great firmware, sadly!). In addition, there are a few switches thrown in to get the 20 connections, but these are transparent.

The reason for the change is my concern that the WRT54G is becoming bogged down with the amount of traffic (connections) it's receiving, especially with bittorrent. It often slows during periods of heavy use, while our bandwidth usage is not highly taxed. I have tried many solutions, from changing QoS rules to simply asking people to cut it out, but to little avail. My users are not tech savvy. I feel our connection should be perfectly capable of handling our traffic, and frankly the situation isn't so bad that it couldn't be left alone, but I've been curious about making a linux router for some time now. I've tried Smoothwall, but I still want more control, and I think I could engineer something lighter (that could run on top of a server or full functional arch box!).

The hardware I intend to use should be highly capable of doing the tasks I expect of it. Personally, I've managed our WRT for some time now, including finding and flashing Tomato onto it, so networking is not unfamiliar to me. I've done quite a bit of research into the topic, but I haven't made anything yet. I'm familiar with the ArchWiki's suggestions ([wiki]Router[/wiki]) on what to use/do, so this isn't intended to be a what-do-I-do, rather a practical question of what-do-you-think-is-best to the Arch community.

So, I'd like my router to have these capabilities:

# firewall (iptables -> arno || shorewall || ufw || firehol)
# serve dhcp ip addresses (dnsmasq || dhcp)
# implement traffic shaping/QoS (tc -> iptables: wondershaper)
# UPnP and port forwarding (gupnp-igd || linuxigd)

As far as I've read, this stuff doesn't require that many packages. Iptables and the kernel tc are incredibly light and powerful. Of course, the tradeoff is that they are absolutely cryptic to use! That's where the scripts come in. I've looked at Arno's firewall scripts, shorewall, ufw, and firehol. Arno's seems to be an arch favorite, and I've heard good things about Firehol. Shorewall is also very attractive. I have not used these yet, I'm curious to see what your opinions are.

So the firewall is handled. I'm more curious about tc/QoS. tc appears more cryptic than even iptables, but finding a good script or method for creating QoS rules is elusive at best. The LARTC site has some information, though I confess to not having read all of it through yet. It can apparently do incredibly complex things; a python interface or script generator would be fantastically awesome. Do any of you use or have ideas on a QoS solution?

A dhcp server appears to be dead simple to setup; I can either user dhcp or dnsmasq for this. Some of the firewalls (like Shorewall) apparently take care of this. UPnP research has uncovered two potentially useful libraries; implementation and use is still a mystery.

Since this might become a full on project, there are a couple of things I think would be incredibly useful, but are not absolutely necessary at this point. It would be nice if it had:

# a webserver (apache || other?)
# secure protocol support (vpn, ssh, etc.)
# wireless functionality
# media server / other server daemons
# fully functional arch box when needed!
# web management interface!

There are quite a few articles online, including here, that deal with creating a router out of a linux box. Collating them is tough, as they often seem to use different pieces of software, or are outdated, etc. the Linux Advanced Routing and Traffic Control (http://lartc.org/) site is great, but also incredibly in depth and lengthy. The above is a compilation of my research into the subject.

Given the focus and expertise of the Arch community, I'd love for this to be a resource to de-mystify the creation of a router. I've seen complex linux topics explained here more simply than anywhere else. Ideally, I'd like to create an approachable resource and set of tools, yet grounded in the elegance of those already available. Any offerings of functional example scripts are highly appreciated!

Thanks for the time it took you to read this; I'll continue updating and posting here as my progress continues. Please do share your input on the situation!

Again, thanks!

Last edited by ninesharp (2010-03-10 16:59:55)

brenix · 2010-03-09 23:56:03

I'll be curious what other suggestions people have to this project and I like it! I've setup an Arch-based router a couple of times (based off the wiki article), and it's definitely worth it. I've never set one up for more than 6 users, but the ones I've setup work flawlessly with 6 users simultaneously.. Not only that, but IMO, its almost easier to manage the DHCP, Firewall, etc than a standard Linksys router. Plus it has much more control/features.

Since most consumer routers don't allow you to tweak a lot of the settings, you can't do much to improve throughput. I've seen a pretty decent throughput increase once I setup an Arch-based router..

ninesharp wrote:

So, I'd like my router to have these capabilities:
# firewall (iptables -> arno || shorewall || ufw || firehol)
# serve dhcp ip addresses (dnsmasq || dhcp || other?)
# implement traffic shaping/QoS (tc -> iptables?)
# UPnP and port forwarding (gupnp-igd || linuxigd ?, see topic "ArchRouter: UPnP")

Firewall: Im not a guru with IPTables, so I've used shorewall and it seems to work great with a small network. The setup and port forwarding is very easy and it seems to be as secure as anything else
DHCP: I havent used anything else other than dnsmasq, but it seems very lightweight.
QOS/UPNP: I haven't needed to use these, but shorewall seems to support both and it looks fairly simple to setup. (See Shorewall's QoS and UPNP guides..

ninesharp wrote:

Since this might become a full on project, there are a couple of things I think would be incredibly useful, but are not absolutely necessary at this point. It would be nice if it had:
# a webserver (apache || other?)
# secure protocol support (vpn, ssh, etc.)
# wireless functionality
# media server / other server daemons
# fully functional arch box when needed!
# web management interface!

I would love to see this as a project! I could almost see an Arch-based router distro. It would be cool to see a web management interface, though I love my command line and haven't really got to setting one up yet . Before setting up my first Arch-based router, I used Pfsense. It was nice and had a web management interface, but it didn't provide the complexity+simplicity that I was looking for...

Well, I don't know if any of this will help, but I'll check back on this thread frequently. If you have configuration questions, let me know!

Edit: Some of the other things I thought of that you could add to the router:
- Web proxy (to save bandwith): Polipo or Squid (I prefer polipo, but squid can run transparently..)
- Denyhosts (If you plan on using ssh it will block all those ssh brute force attacks)
- Cacti or Ntop: (web based network/bandwidth usage monitoring +more!..)
- With DNSMasq or a web proxy, you can also block ads/domains/websites

Last edited by brenix (2010-03-10 00:20:12)

ninesharp · 2010-03-10 20:29:00

Thanks Brenix! Your input about Shorewall has been very helpful, and I think I'm going to give it a test-run tonight.

I'm also going to give arno's and firehol a try too; they seem quite powerful in their own right. It looks to be that doing the QoS/TC I want to is going to require some iptables stuff anyways; the complex Shorewall tc configuration seems to approach that syntax the more detailed you get. Thankfully, it doesn't actually look that bad when I went over it. It's probably best that I learn iptables anyways, or at least some frontends. Wondershaper is also mentioned on the LARTC site, or at least used to be. It seems to be a widely used tool. I'll see what that's like, and continue looking for tc rules scripts. Concerning UPnP, it appears that Shroewall uses linuxigd. This will be my default since it is more documented than gupnp-igd as far as I know.

Ntop and cacti are great suggestions! I forgot to include tools like that in the list, though they will be very useful. I'll look into a proxy too; I've never used one, but it could help in this situation. Certainly a good package to include for the project.

Thanks!

Last edited by ninesharp (2010-03-10 20:32:50)

Stythys · 2010-03-10 21:01:49

reminds me of a distro I came across awhile back: http://www.engardelinux.org. development seems to have slowed, though...

I would fully support an arch-based router distro . Hit me up if you need any hosting help.

Last edited by Stythys (2010-03-10 21:15:42)

Sin.citadel · 2010-03-11 13:55:20

i currently am using arch as a router to provide connectivity for ~ 60 users, i use dnsmasq for dns/dhcp and squid for transparent proxying with QoS rules set up for both squid and all other routable traffic, i also use ssh as well as MySQL and apache. arch works wonderfully considering the machine is only a 1.8 Ghz P4 with 256MB of RAM. if you need my QoS file and the iptables file with it, i'll post it here.

madeye · 2010-03-11 15:43:29

This is cool. I've played with the idea myself, but I just can't find the time. It will be interesting to see what this evolves into.
I used Mandrake 7.2 SNF and 8.0 MNF way back when it was available. It used shorewall and a web interface the mandrake people (now mandriva) made.
I don't know if it's GPL'ed, but I have the CDs lying here somewhere. If you are interested in checking out the interface just let me know.

I will be following this thread too. Maybe I can help along the way. If nothing else I can help test it.

hatten · 2010-03-11 18:21:06

Cool idea, not something that's viable for me atm, but I wish you luck.

*subscribing*

d4g0n · 2010-03-13 12:11:42

I am also subscribing. This is interesting

Fungyo · 2010-03-20 04:33:47

I'm looking at setting up an Arch router myself. Hopefully I can help.

mikesd · 2010-03-20 08:21:42

Interesting. I run Debian on my router but have been thinking of replacing it with Arch at some point.

dnsmasq is a nice package. Makes it easy to handle dhcp and dns all in one go. Local names are dead easy to setup too, just add them to the /etc/hosts file.

Quite a few of our older, hand rolled, routers at work use shorewall and it is pretty nice. I just use iptables at home though.

QOS is one of those things I always wanted to setup but never got around to it.

If you want a ready to go Linux based router check out Vyatta[1] and Endian[2]. Both of these are pretty neat. Endian features content filtering, which we need, otherwise I would go with Vyatta.

[1] http://www.vyatta.com/
[2] http://www.endian.com/

xCrucialDudex · 2010-04-12 18:50:17

Sin.citadel wrote:

i currently am using arch as a router to provide connectivity for ~ 60 users, i use dnsmasq for dns/dhcp and squid for transparent proxying with QoS rules set up for both squid and all other routable traffic, i also use ssh as well as MySQL and apache. arch works wonderfully considering the machine is only a 1.8 Ghz P4 with 256MB of RAM. if you need my QoS file and the iptables file with it, i'll post it here.

I'd love to take a look at your configuration if you don't mind. I've written one myself today but my setup is rather complex and wouldn't probably do for a good illustration without extensive notes. Something I just don't have time for at the moment. If you can, please, share your QoS configuration files. In fact, it's pretty hard to come across a cohesive, comprehensive real-life example even though tc have been around for years.

Sin.citadel · 2010-04-13 17:33:40

Here is my configuration, it uses htb, i wrote this config about a year ago and havent looked at it for quite a while, so there might be more improvements that can be added to it, if someone can tell me what i should change to get more out of this, it'll really help me out.

#!/bin/bash

declare -r DEV="eth0"
declare -r MAX_CEIL="4.1mbit"
tc qdisc del dev $DEV root

# TC Script below is for Download Limitations only

tc qdisc add dev $DEV root handle 1: htb default 15 
tc class add dev $DEV parent 1: classid 1:1 htb rate $MAX_CEIL ceil $MAX_CEIL
# Class of Low Latency Data
tc class add dev $DEV parent 1:1 classid 1:10 htb rate 768kbit ceil 768kbit prio 1

# Class for HTTPS Traffic 
tc class add dev $DEV parent 1:1 classid 1:11 htb rate 256kbit ceil 1.5mbit prio 2

# Class for HTTP Traffic
tc class add dev $DEV parent 1:1 classid 1:12 htb rate 2.25mbit ceil $MAX_CEIL prio 3

# Class for Other Data
tc class add dev $DEV parent 1:1 classid 1:15 htb rate 768kbit ceil $MAX_CEIL prio 5

# Add sfq for Fair Distribution
tc qdisc add dev $DEV parent 1:10 handle 120: sfq perturb 10
tc qdisc add dev $DEV parent 1:11 handle 130: sfq perturb 10
tc qdisc add dev $DEV parent 1:12 handle 140: sfq perturb 10

# Class For Un-Filtered Data
tc class add dev $DEV parent 1: classid 1:20 htb rate 90mbit ceil 90mbit 

# Set up Filters
#
# Filter for Un-Filtered Data
tc filter add dev $DEV parent 1:0 protocol ip prio 0 handle 20 fw classid 1:20
tc filter add dev $DEV parent 1:0 protocol ip prio 0 handle 10 fw classid 1:20
# Filter for Low Latency Data
tc filter add dev $DEV parent 1:0 protocol ip prio 1 handle 11 fw classid 1:10
# Filter For HTTPS Traffic
tc filter add dev $DEV parent 1:0 protocol ip prio 1 handle 12 fw classid 1:11
# Filter for HTTP Traffic
tc filter add dev $DEV parent 1:0 protocol ip prio 3 handle 13 fw classid 1:12
# Filter for Other Data
tc filter add dev $DEV parent 1:0 protocol ip prio 5 handle 16 fw classid 1:15

ninesharp · 2010-05-28 13:54:24

Thanks for the config S.C! That looks simple enough to tweak and suit my needs. I'll post mine once I get it all running.

Speaking of, I apologize for the lack of updates. I ran out of time to put toward this for a while, but it's surfaced on the priority queue again! Hopefully a working model will appear soon.

I decided on going straight into iptables; the front ends are worth investigating when I have more time but if I'm going to learn a new syntax anyways, it might as well be iptables. S.C's config above gives me hope that the iptables/tc config won't be terribly difficult for the simple things that I'm doing.

So, my setup to begin with, let's call it 'alpha', will be just a simple dnsmasq, iptables, and tc config; which I'll post when completed.

Thanks for the ongoing interest in this topic!

Arch Linux

#1 2010-03-09 22:18:57

ArchRouter

#2 2010-03-09 23:56:03

Re: ArchRouter

#3 2010-03-10 20:29:00

Re: ArchRouter

#4 2010-03-10 21:01:49

Re: ArchRouter

#5 2010-03-11 13:55:20

Re: ArchRouter

#6 2010-03-11 15:43:29

Re: ArchRouter

#7 2010-03-11 18:21:06

Re: ArchRouter

#8 2010-03-13 12:11:42

Re: ArchRouter

#9 2010-03-20 04:33:47

Re: ArchRouter

#10 2010-03-20 08:21:42

Re: ArchRouter

#11 2010-04-12 18:50:17

Re: ArchRouter

#12 2010-04-13 17:33:40

Re: ArchRouter

#13 2010-05-28 13:54:24

Re: ArchRouter

Board footer