[Maarten Van Horenbeeck] [Information Security] [Resources]

Establishing a VPN between OpenSWAN and a PIX firewall

The Cisco PIX is a powerful hardware firewall solution sold by Cisco Systems. OpenSWAN is an opensource project enabling IPSEC functionality on the Linux platform. This document is merely a quick summary of what I did to get a VPN established between OpenSWAN and the PIX. Interesting about OpenSWAN as a solution is that it does not require a kernel patch if you do not require NAT traversal. The loadable module ipsec.o is all you need.

First, get a recent OpenSWAN release and install it using its documentation (this will mainly be a make && make programs install). Afterwards reboot or load the ipsec.o module Make sure the firewall on your OpenSWAN device is allowing port 500/udp and proto 50 (esp) from the IP address of your PIX firewall. Next, create the configuration file ipsec.conf so that it contains the following items. The comments are ofcourse not required. The important aspect here is that we added a non-default line spi=0x0. This sets the security parameter index, that identifies which SA to use. This is not a requirement, but could help in some situations where your VPN terminator is firewalled to build more stringent rulesets. Also note that the line auto=add specifies that OpenSWAN should accept incoming connections, but not initiate them.

version 2.0

config setup
       interfaces=%defaultroute
       klipsdebug=all #enable debugging
       plutodebug=all

conn tunnelipsec
       type=tunnel     #tunnel mode ipsec
       left=x.x.x.x   #the IP address of your OpenSWAN endpoint
       leftnexthop=y.y.y.y   #default gateway
       leftsubnet=10.0.0.0/8   # network behind your endpoint
       right=%any   # Your IP, or %any for a road-warrior setup
       rightnexthop=%defaultroute    #defaultroute for road warrior unknown
       rightsubnet=10.2.0.0/16    #network behind the PIX
       esp=3des-sha1    #esp: 3des, hmac: sha1
       keyexchange=ike    #use regular ike
       authby=secret    #pre-shared secret,  you can also use rsa nounces
       pfs=yes    #use perfect forward secrecy
       auto=add     #don't initiate tunnel, but allow incoming
       spi=0x0    #use base spi of 0x0 for PIX

#following items disable opportunistic encryption (otherwise too much connection time would be spilled on something which is not used in real-life)

conn block
 auto=ignore

conn private
 auto=ignore

conn private-or-clear
 auto=ignore

conn clear-or-private
 auto=ignore

conn clear
 auto=ignore

conn packetdefault
 auto=ignore

Now create a /etc/ipsec.secrets file that contains the IP address of your OpenSWAN terminator, the PIX firewall, and the preshared key. An example:
x.x.x.x y.y.y.y : PSK "***************"
The ****** is the preshared key, x.x.x.x is het IP address of your OpenSWAN terminator, while y.y.y.y is that of the PIX. If the PIX firewall has a dynamic IP, enter %any.

Next, we configure the PIX firewall. Here no real exceptions apply, although you should not forget to enable PFS (Perfect Forward Secrecy). If this is not included, while it is on the OpenSWAN server, the VPN will only be able to establish in one direction.

sysopt connection permit-ipsec
crypto ipsec transform-set vpnset esp-3des esp-sha-hmac
crypto map vpnmap 10 ipsec-isakmp
crypto map vpnmap 10 match address 108
crypto map vpnmap 10 set peer x.x.x.x 
crypto map vpnmap 10 set transform-set vpnset
crypto map vpnmap 10 set pfs group2
crypto map vpnmap interface outside
isakmp enable outside
isakmp key ******** address x.x.x.x netmask 255.255.255.255 no-xauth no-config-mode
isakmp identity address
isakmp policy 5 authentication pre-share
isakmp policy 5 encryption 3des
isakmp policy 5 hash sha
isakmp policy 5 group 2
isakmp policy 5 lifetime 3600
nat (inside) 0 access-list 108
access-list 108 permit ip 10.2.0.0 255.255.0.0 10.0.0.0 255.0.0.0 
And that is it. When you now create so-called "meaningful traffic", meaning traffic that matches access-list 108, the VPN should establish perfectly.