IPFW with FireHOL block lists

The FireHOL project, another firewall and qos project, compiles a set of ip block lists. There are a number of lists that have different focuses - there is some overlap between them. These could be handy to use with IPFW. So let's see what we can do…

We have some constraints:

• We will want to update the lists periodically without causing a disruption
• These lists can be large
• If we use more than one lists, we should eliminate the overlap

Tables are going to make this idea possible and down right simple.

IPFW Lookup Tables have some handy features:

• They're fast
• They can be swapped around
• The can have elements added/removed without having to reload the firewall rules

When we build our IPFW rules we're going to create a firehol table. We will deny packets that on the WAN interface that are contained in that table. Just to be safe we will also create a table of addresses that we always want to trust. Addresses in that table will skip around the firehol deny rule - so something like like this…

TBL_IP="table(ok)"
ipfw table ok create
ipfw table ok add $WAN_MASK
ipfw table ok add $ISPGW
ipfw table ok add mydomain.example
ipfw table ok add myotherdomain.example
ipfw table ok add $DNS1
ipfw table ok add $DNS2

TBLE_FIREHOL="table(firehol)"
ipfw table firehol create missing

LIST=/etc/ipfw/firehol/list.combined
if [ -e $LIST ]; then
   echo "load bad guys table"
   /etc/ipfw/firehol/fireload.pl $LIST > /etc/ipfw/firehol/log.load 2>&1
else
   echo "WARNING: $LIST not found"
fi

The fireload.pl script does what it says on the tin, we will talk a bit more about it in a bit.

Later in the rule set we will have something like…

ipfw add 1036 skipto 1040 ip4 from $TBL_OK to any           in  recv $WAN
ipfw add 1037 skipto 1040 ip4 from any     to $TBL_OK       out xmit $WAN
ipfw add 1038 deny ip4 from $TBL_FIREHOL   to any           in  recv $WAN
ipfw add 1039 deny ip4 from any            to $TBL_FIREHOL  out xmit $WAN
ipfw add 1040 count 

The lists consist of comment lines, CIDR notation subnet masks, or individual addresses.

We're going to use a fairly naive approach to keep things simple.

1. Download the lists we want (ie firehol_level1, firehol_level2, firehol_level3)
2. Combine them
3. Sort the combined list and remove duplicates (ie sort -u -o combinedlist combinedlist)
4. Load them into a temporary table
5. Swap the temporary table the main table

I use a set of perl scripts for this but it could be done in any number of ways. There's nothing very fancy here. We're going to grab the lists using something like curl, we then concatenate the lists removing any none CIDR/IP address lines. I prefer, for no great reason, to have all the addresses in CIDR format so I convert the IP addresses into a CIDR netmask by appending /32 to it. The combined list is then sorted removing non-unique lines.

This is where the fireload.pl script comes in that was referenced above. Again, there's nothing really fancy here so you could do it with a shell script just as well. We create a temporary table, add each CIDR address from our combined list to that table and then swap it in. Remember that we're working with lookup tables so we have to add a key and a value. The key would be the CIDR address, the value can always be 0. 0 is assumed if not given. The script would need to execute commands like…

ipfw table fireholload create missing
ipfw table fireholload add 0.0.0.0/8
ipfw table fireholload add 10.0.0.0/8
ipfw table fireholload add 192.168.0.0/16
:
ipfw table fireholload add 224.0.0.0/3
ipfw table fireholload swap firehol
ipfw table fireholload destroy

And we're done. The new combined list is loaded into the firehol table that our rule set is making use of.

Note that the current combination of lists 1, 2, and 3 with duplicates removed contains over 35,000 CIDR/IP addresses so the above is not the best strategy. Fortunately we can add more than a single address to the table at a time so we can chunk things up a bit and do, say, 50 at a time. This speeds things up slightly. The only difference is that we MUST provide a value this way so the above adds would look like

ipfw table fireholload add 0.0.0.0/8 0 10.0.0.0/8 0 192.168.0.0/16 0 224.0.0.0/3 0

Note that while we are removing exact duplicates we're not removing subnets, so if 10.0.0.0/8 is in the combined list, 10.10.0.0/16 or 10.20.20.20 might also be in the list. There are a number of tools and strategies that could be used to overcome this but it's just not worth the effort. The lookup tables are very fast so the reduction in entries would be little or nothing.