Cisco VPN debugging by crystal ball

nospam@example.com (Ralf Ertzinger) — Mon, 18 Jan 2010 10:37:14 +0100

In the hope that google picks this up:

The problem space is a Cisco PIX terminating an IPSec VPN tunnel with a Checkpoint firewall on the other end. The tunnel does not work (the phase 2 setup fails). The Cisco logs the following debug messages:

ISAKMP (0): processing SA payload. message ID = 1911693629
ISAKMP : Checking IPSec proposal 1
ISAKMP: transform 1, ESP_3DES
ISAKMP:   attributes in transform:
ISAKMP:      SA life type in seconds
ISAKMP:      SA life duration (VPI) of  0x0 0x0 0xe 0x10
ISAKMP:      authenticator is HMAC-SHA
ISAKMP:      encaps is 1
ISAKMP (0): atts are acceptable.
ISAKMP : Checking IPSec proposal 1
ISAKMP (0): atts not acceptable. Next payload is 0
ISAKMP (0): SA not acceptable!
ISAKMP (0): sending NOTIFY message 14 protocol 0
return status is IKMP_ERR_NO_RETRANS

The log message above was created by an incoming proposal (the remote end proposed a connection to the Cisco PIX). This is useless and confusing at the same time. An IPSec proposal contains a list of parameters, sent by one end of the connection, specifying the parameters it is willing to use to establish a secure connection. This proposal specifies 3DES as the encryption algorithm, SHA as a hash function, and a lifetime for the connection of 3600 seconds (after which the connection has to be renegotiated).

As can be seen, the PIX accepts this proposal (as it should), since these parameters match those configured on the PIX for this connection. It then goes on to check the same proposal again, just to reject it this time.

The completely non-obvious solution to this is to disable compression (which the PIX does not support) on the Checkpoint. Why the PIX is unable to even give me a hexdump of the offending parameter in the proposal I’ll probably never know.

For the good of all of us - Cisco

Cisco VPN debugging by crystal ball