Site to Site IPSEC problems

[Dizzy_Read]

Hi!

We have some problem with routing? via IPSec site to site

1. 2 ISP on both side, 4 ipsec tunnels from Branch site (60c) to mail office (1000c).

2. Phase 2 on all tunnels 0.0.0.0/0.0.0.0 as networks, static routes with different distance (1-2-3-4) for IPSEC failover.  

3. "LAN" 192.168.7.0/24 on 60С and "LAN" 192.168.0.0/24 on 1000C

4. FW on 60c and 1000c 5.2.11 latest

5. All work fine, traffic flow from any LANs to any LANs via IPSec BUT ...

If trace route from LAN 192.168.0.0 host to host in 192.168.7.0 i see, WAN1 IP of 60C in trace route ....

1    <1 ms    <1 ms    <1 ms  192.168.0.251 (1000C LAN IP) 2     3 ms     3 ms     2 ms  xx.xx.xx.xx (60C WAN1 IP) 3     3 ms     2 ms     2 ms  host5 [192.168.7.1] (host in 192.168.7.0 network)

Why traffic flow like this ??? We have few others devices connected same way - no tracert problems!!!

We try check any settings - all the same, only difference - connected ports. We think - traffic flow as ifindex of port on FGT ????

How we can resolve issue?

We have troubles with applications like TMG 2010 - client source IP (WAN IP 60c) unreachable in logs ...

Tnx for any help! 

[Dizzy_Read]

Antonio, tnx again for help ! )))   MTU test to problem side (192.168.1.0/24 network)   mturoute.exe -t 192.168.1.1 mturoute to 192.168.1.1, 30 hops max, variable sized packets * ICMP Fragmentation is not permitted. * * Speed optimization is enabled. * * Maximum payload is 10000 bytes. *  1  +-  host: 192.168.0.251  max: 1500 bytes  2  No response from traceroute for this TTL.  Tried 3 times  - WHY ?  3  -+++++-+...-++++  host: 192.168.1.1  max: 1444 bytes   D:\mturoute.exe 192.168.1.1 * ICMP Fragmentation is not permitted. * * Speed optimization is enabled. * * Maximum payload is 10000 bytes. * - ICMP payload of 1472 bytes is too big. + ICMP payload of 92 bytes succeeded. + ICMP payload of 782 bytes succeeded. + ICMP payload of 1127 bytes succeeded. + ICMP payload of 1299 bytes succeeded. + ICMP payload of 1385 bytes succeeded. - ICMP payload of 1428 bytes is too big. + ICMP payload of 1406 bytes succeeded. ...- ICMP payload of 1417 bytes failed. (IP_REQ_TIMED_OUT) - ??? + ICMP payload of 1411 bytes succeeded. + ICMP payload of 1414 bytes succeeded. + ICMP payload of 1415 bytes succeeded. + ICMP payload of 1416 bytes succeeded. Path MTU: 1444 bytes.   MTU test to NO problem side (192.168.5.0/24 network)   D:\mturoute.exe -t 192.168.5.1 mturoute to 192.168.5.1, 30 hops max, variable sized packets * ICMP Fragmentation is not permitted. * * Speed optimization is enabled. * * Maximum payload is 10000 bytes. *  1  +-  host: 192.168.0.251  max: 1500 bytes  2  -+++++-++--+  host: 192.168.5.254  max: 1446 bytes  3  -+++++++++++-  host: 192.168.5.1  max: 1444 bytes   mturoute.exe 192.168.5.1 * ICMP Fragmentation is not permitted. * * Speed optimization is enabled. * * Maximum payload is 10000 bytes. * - ICMP payload of 1472 bytes is too big. + ICMP payload of 92 bytes succeeded. + ICMP payload of 782 bytes succeeded. + ICMP payload of 1127 bytes succeeded. + ICMP payload of 1299 bytes succeeded. + ICMP payload of 1385 bytes succeeded. - ICMP payload of 1428 bytes is too big. + ICMP payload of 1406 bytes succeeded. - ICMP payload of 1417 bytes is too big. + ICMP payload of 1411 bytes succeeded. + ICMP payload of 1414 bytes succeeded. + ICMP payload of 1415 bytes succeeded. + ICMP payload of 1416 bytes succeeded. Path MTU: 1444 bytes.   Effective MTU 1444 ?  I need to lower it on FGT (as i know, i can't do it on ipsec interface) or on WinSrv ? 

[Antonio_Milanese]

Hi Dizzy,

ok.. so the MTU it's lowered only by ipsec overhead..

diag vpn tunnel list name YOU_VPN_NAME | grep mtu

should enlist a 1443 mtu for the tunnel

changing MTU or checking PDMTU it's  into the registry of TMG machine may work..but a reboot it's required..

On tunnel interfaces you cannot ovveride mtu as You have found but You can at least clamp mss ( or using per policy adjustment) so I'll give it a try.

Moreever use wireshark at TMG to check negotiated mss during syn/syn-ack to confirm that it's correctly negotiated: may be you can check before set mss clamping and after so you have a indirect confirmation that the problem was related to mtu.

Regards,

Antonio

p.s. i should have noted beferehand... disable ALL tcp offload engine on TMG as per best practices.. it's a real pain with TMG and in general with everything before 2012R2.

[Dizzy_Read]

Antonio, tnx again for help ! )))

MTU test to problem side (192.168.1.0/24 network)

mturoute.exe -t 192.168.1.1 mturoute to 192.168.1.1, 30 hops max, variable sized packets * ICMP Fragmentation is not permitted. * * Speed optimization is enabled. * * Maximum payload is 10000 bytes. *  1  +-  host: 192.168.0.251  max: 1500 bytes  2  No response from traceroute for this TTL.  Tried 3 times  - WHY ?  3  -+++++-+...-++++  host: 192.168.1.1  max: 1444 bytes

D:\mturoute.exe 192.168.1.1 * ICMP Fragmentation is not permitted. * * Speed optimization is enabled. * * Maximum payload is 10000 bytes. * - ICMP payload of 1472 bytes is too big. + ICMP payload of 92 bytes succeeded. + ICMP payload of 782 bytes succeeded. + ICMP payload of 1127 bytes succeeded. + ICMP payload of 1299 bytes succeeded. + ICMP payload of 1385 bytes succeeded. - ICMP payload of 1428 bytes is too big. + ICMP payload of 1406 bytes succeeded. ...- ICMP payload of 1417 bytes failed. (IP_REQ_TIMED_OUT) - ??? + ICMP payload of 1411 bytes succeeded. + ICMP payload of 1414 bytes succeeded. + ICMP payload of 1415 bytes succeeded. + ICMP payload of 1416 bytes succeeded. Path MTU: 1444 bytes.

MTU test to NO problem side (192.168.5.0/24 network)

D:\mturoute.exe -t 192.168.5.1 mturoute to 192.168.5.1, 30 hops max, variable sized packets * ICMP Fragmentation is not permitted. * * Speed optimization is enabled. * * Maximum payload is 10000 bytes. *  1  +-  host: 192.168.0.251  max: 1500 bytes  2  -+++++-++--+  host: 192.168.5.254  max: 1446 bytes  3  -+++++++++++-  host: 192.168.5.1  max: 1444 bytes

mturoute.exe 192.168.5.1 * ICMP Fragmentation is not permitted. * * Speed optimization is enabled. * * Maximum payload is 10000 bytes. * - ICMP payload of 1472 bytes is too big. + ICMP payload of 92 bytes succeeded. + ICMP payload of 782 bytes succeeded. + ICMP payload of 1127 bytes succeeded. + ICMP payload of 1299 bytes succeeded. + ICMP payload of 1385 bytes succeeded. - ICMP payload of 1428 bytes is too big. + ICMP payload of 1406 bytes succeeded. - ICMP payload of 1417 bytes is too big. + ICMP payload of 1411 bytes succeeded. + ICMP payload of 1414 bytes succeeded. + ICMP payload of 1415 bytes succeeded. + ICMP payload of 1416 bytes succeeded. Path MTU: 1444 bytes.

Effective MTU 1444 ?  I need to lower it on FGT (as i know, i can't do it on ipsec interface) or on WinSrv ?