[TriLUG] linux routing example

[Jason Tower]

i ran across an interesting situation recently at a client site that i 
thought i'd share with the list.  this particular client is running a 
fairly active webserver (using red hat), so much so that they decided 
to add a second T1 solely for web traffic, instead of letting their 
existing T1 handle both web server and LAN traffic.  the catch was, 
they wanted to have zero downtime when they made the DNS changes so the 
web server had to function properly on both T1s simultaneously (the two 
T1s were from different ISPs and had totally different subnets).  this 
is harder than it seems at first glance; in order for this to function 
you have to make sure that traffic "leaves" on the same interface it 
arrived on, regardless of which route is preferred.  simply setting 
gateways for each interface does not work.  the solution was to take a 
page from the linux advanced routing howto (special thanks to jeremy 
portzer for pointing them out):

http://www.tldp.org/HOWTO/Adv-Routing-HOWTO/

particularly the part on utilizing multiple uplinks, or split access:

http://www.tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.rpdb.multiple-links.html

i ended up using a small shell script that is called from rc.local at 
boot time (the IP addresses have been randomized):
-----------------------------------------------------
#!/bin/sh
IF1=eth0
IF2=eth1
IP1=66.45.113.214 #IP address of eth0
IP2=133.37.29.34 #IP address of eth1
P1=66.45.113.213 #gateway for eth0
P2=133.37.29.33 #gateway for eth1
P1_NET=66.45.113.209 #network of eth0
P2_NET=133.37.29.30 #networkof eth1

ip route add $P1_NET dev $IF1 src $IP1 table T1
          ip route add default via $P1 table T1
          ip route add $P2_NET dev $IF2 src $IP2 table T2
          ip route add default via $P2 table T2

ip route add $P1_NET dev $IF1 src $IP1
            ip route add $P2_NET dev $IF2 src $IP2

ip route add default via $P2

ip rule add from $IP1 table T1
            ip rule add from $IP2 table T2
--------------------------------------------------
finally, add two routing table entires into /etc/iproute2/rt_tables:

echo "200 T1">>/etc/iproute2/rt_tables
echo "201 T2">>/etc/iproute2/rt_tables

once this was done the web server worked flawlessly, serving up pages to 
hosts regardless of whether they used the original IP address or the 
new one.  this way it didn't matter how long DNS changes took to 
propogate, both results worked equally well.

the iproute2 tools and utilities is amazingly powerful, and are capable 
of a lot more than what is shown here.  have fun with it!

jason