The information in the below guide turned out to be surprisingly difficult to come by, it took days of googling before everything was running smoothly.
The setup in my case is two Ubuntu 12.04 boxes (Dell R420s) acting as firewalls / load balancers in front of two switches which in turn connects to an arbitrary amount of boxes behind them (Dell R820s) which are also running Ubuntu 12.04.
All boxes have two network cards, eth0 and eth1. When it comes to the boxes on the LAN, they are connected to switch 1 on eth0 and switch 2 on eth1 for redundancy / failover (more on that in a future post). The firewall boxes are connected through eth0 to the gateway and to their respective switch on eth1.
So the basic problem is to setup the boxes which are acting as firewalls so that they route information sent to an external IP to the internal LAN IP, the TCP packets going out also need to look like they’re coming from the correct external IP. Furthermore, the machines on the LAN will also initiate TCP communication so their route needed to be changed to the correct firewall machine.
With that in mind, let’s get to it, it’s surprisingly little work once you actually know how it’s done, let’s walk through the setup of one of the firewall boxes, the other one will be setup virtually identically so it’s enough with walking through just one of them (comments on how to setup the other one are at the end):
1.) On the firewall box, enable forwarding by adding net.ipv4.ip_forward=1 to /etc/sysctl.conf and running sysctl -p. That will both enable it in the current session and after subsequent reboots.
2.) Given that the firewall box’s external IP is 126.96.36.199 and the external IPs that the R820s will listen to are 188.8.131.52 and 184.108.40.206 we’ve got the following /etc/network/interfaces setup on the firewall box:
auto lo iface lo inet loopback auto eth0 iface eth0 inet static address 220.127.116.11 auto eth0:0 iface eth0:0 inet static address 18.104.22.168 auto eth0:1 iface eth0:1 inet static address 22.214.171.124 auto eth1 iface eth1 inet static address 10.40.10.253 netmask 255.255.255.0
So, we’ve got the external IPs attached to virtual interfaces, eth0:0 and eth0:1. Don’t forget to ifup eth0:0 and ifup eth0:1 after you’ve added them. Note the internal IP on eth1 through which we can access the box on the LAN: 10.40.10.253.
3.) To be able to for instance run wget somepage.com and have it work you need to change the routing on each LAN box so that it goes through the firewall box: ip route change default via 10.40.10.253 dev eth0.
4.) Finally, let’s take a look at the complete iptables configuration which can be imported and exported through commands which have previously been explained.
*nat -A PREROUTING -d 126.96.36.199/32 -i eth0 -j DNAT --to-destination 10.40.10.240 -A PREROUTING -d 188.8.131.52/32 -i eth0 -j DNAT --to-destination 10.40.10.243 -A POSTROUTING -s 10.40.10.240/32 -j SNAT --to-source 184.108.40.206 -A POSTROUTING -s 10.40.10.243/32 -j SNAT --to-source 220.127.116.11 COMMIT *filter -A INPUT -i lo -j ACCEPT -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p tcp -m tcp --dport 22 -m state --state NEW -j ACCEPT -A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT -A INPUT -p tcp -m tcp --dport 443 -m state --state NEW -j ACCEPT -A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT -A INPUT -j DROP -A FORWARD -s 18.104.22.168/32 -p tcp -m tcp --dport 3306 -j ACCEPT -A FORWARD -p tcp -m tcp --dport 3306 -j DROP -A FORWARD -s 10.40.10.240/32 -j ACCEPT -A FORWARD -s 10.40.10.243/32 -j ACCEPT -A FORWARD -d 10.40.10.240/32 -j ACCEPT -A FORWARD -d 10.40.10.243/32 -j ACCEPT -A FORWARD -i eth1 -j ACCEPT -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -j ACCEPT -A OUTPUT -o lo -j ACCEPT -A OUTPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT COMMIT
Let’s start from the top with the *nat section. The internal IPs connected to eth0 on the R820s are 10.40.10.240 and 10.40.10.243. To correctly translate / route incoming requests to each R820 we need to translate each packet through DNATing, this is done in the first two PREROUTING lines. This makes it look like an external request to the external IP is actually aimed at the internal IP, without this the switch wouldn’t know what to do.
When the machine subsequently replies we need to translate back, this is done through the POSTROUTING lines.
In the *filter section we first open up SSH, HTTP, HTTPS and pinging. More important are all FORWARD lines, some of them might or might not be superfluous but the above works so I don’t want to mess with it more by testing which lines are unnecessary (if any) and which lines are not.
Note that we disable forwarding to MySQL on port 3306, only communication from the 22.214.171.124 IP is allowed to come through.
5.) Given that the R820 LAN IPs above are on eth0 you only need to replace them with the eth1 equivalent to setup the other firewall box. Remember it’s connected through switch 2 which is connected to each R820’s eth1.