[TriLUG] BSD/Linux firewall with multiple ISP and failover?

[Aaron S. Joyner]

Greg Brown wrote:

>Hey all.  I think I finally hit a dead-end with M0n0wall.  My outer banks
>client now requires fail over from the DSL Internet connection to a 2nd ISP,
>probably a cable modem.  My research indicates that M0n0 doesn't support
>this yet.
>
>What are my options here besides something like a Cisco 2621xm (WAY too
>expensive for this client)?  Does anyone know of an inexpensive appliance
>that they have tested for fail over Internet?  Or can a BSD/Linux box be
>built for this purpose?
>
>Greg
>  
>
As Jon mentioned, it doesn't sound like CARP is the solution for you.  
CARP would allow you to build a second firewall, which would serve to 
provide reliability in case of local machine failure.  It's particularly 
useful because it shares state tables between the two firewalls, so that 
it one dies, connections don't drop, etc and the firewalls can be used 
in an active/active setup, to benefit from double the bandwidth.  This 
is a good goal, but it's really seperate from the root problem you're 
trying to solve.

You want something that allows you to have multiple paths to the 
internet, and should one of those paths die, you want to switch to using 
the alternate path.  This is actually a very easy thing to do, and only 
requires a second ethernet interface in the firewall in question (note 
the word interface, not network card, as technically this could be done 
with a managed switch, vlans, and some craziness if you want to keep 
your existing hardware platform).  In short bullet points, assuming you 
want to use Linux, it'll go something like this:

1 - Get one ISP working, dhcp, whatever is required.  Shutdown that 
interface.
2 - Setup the second interface, get that ISP's connection working, shut 
that interface down.
3 - Pick a few reliable hosts on the internet, I'd choose 6, to use as a 
measure of connectivity.
4 - Configure DHCP on the backup internet connection not to write the 
default gateway or resolv.conf.  It helps if this connection has a 
static ip / default gateway.
5 - Bring up both interfaces, and things should work as expected.  Note, 
you won't properly respond to traffic on the secondary interface, but 
having that interface turned up shouldn't interfere with the primary 
interface.
6 - Setup iproute policy routing such that traffic leaving your 
secondary interface has the gateway set for the secondary default 
gateway, and if your primary is also static you may be able to do the 
same for the primary ISP, or at worst you can leave it in the default 
table.  This is a common technique for multi-homed servers, see here: 
http://www.linuxjournal.com/article/7291 for more information.  This was 
the first google result for the query ["ip rule" multihomed], fell free 
to look for other sources of how to setup multihomed servers to get a 
better feel for using the 'ip rule' and 'ip table' set of commands.  
Reading to have a thorough understanding of these topics is required for 
you to complete steps 7 and 8.
7 - Setup custom "ip rule"s to each of your test hosts, to ensure that 
traffic to that test host goes over the correct interface.
8 - Write a short script which attempts to connect to each of your 
primary isp's test hosts to validate that connection is valid.  If those 
tests fail, try the secondary isp's test hosts, if those succeed, change 
the default 'ip rule' to point t othe other table (see docs referenced 
in step 6 for more detail).

Come back and post again if you can't get it working correctly.  :)

Good luck Greg,
Aaron S. Joyner