Fortigate sending IAX2 packets to its default gateway, not to IPSec tunnel

[batbayar1001]

[/size]Hi,[/size]

I'm new to the Fortinet product, actually it is my first Fortigate firewall configuration.

I configured IPSec tunnel between Cisco RV042 at head office (HQ) and FortiWiFi 30E at branch office (BO) to connect 2 Asterisk servers. 

The HQ subnets are 192.168.16.0/255.255.240.0, BO subnet is 192.168.0.0/255.255.255.224. The tunnel is up and running. I can ping to the servers and access the servers via SSH. Servers can ping each other.

Strange thing is, Fortigate does not forward IAX2 packets between offices via IPSec tunnel.

diag sys session list command shows Fortigate sends IAX2 packets from BO to HQ to its default gateway, not to the tunnel.

[size="2"]

session info: proto=17 proto_state=00 duration=336 expire=178 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=5
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=may_dirty ndr app_valid
statistic(bytes/packets/allow_err): org=3906/84/1 reply=1008/18/1 tuples=3
tx speed(Bps/kbps): 11/0 rx speed(Bps/kbps): 2/0
orgin->sink: org pre->post, reply pre->post dev=15->25/25->15 gwy=gatewayIP/192.168.0.28
hook=post dir=org act=snat 192.168.0.28:4569->192.168.20.253:4569(wanIP:64985)
hook=pre dir=reply act=dnat 192.168.20.253:4569->wanIP:64985(192.168.0.28:4569)
hook=post dir=reply act=noop 192.168.20.253:4569->192.168.0.28:4569(0.0.0.0:0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=000000be tos=ff/ff app_list=2001 app=16841 url_cat=0
dd_type=0 dd_mode=0
total session 1

[/size]

app=16841 is IAX2.

Debug messages:

[size="2"]

2017-04-24 18:28:00 id=20085 trace_id=19 func=print_pkt_detail line=5319 msg="vd-root received a packet(proto=17, 192.168.0.28:4569->192.168.20.253:4569) from internal. "
2017-04-24 18:28:00 id=20085 trace_id=19 func=resolve_ip_tuple_fast line=5394 msg="Find an existing session, id-000000be, original direction"
2017-04-24 18:28:00 id=20085 trace_id=19 func=ids_receive line=281 msg="send to ips"
2017-04-24 18:28:00 id=20085 trace_id=19 func=__ip_session_run_tuple line=3126 msg="SNAT 192.168.0.28->wanIP:64985"

[/size]

Why it sends packet to the IPS when IPS is turned off?

But after I execute diag sys session clear command, the servers starts talking, and the diag sys session list command shows different result.

session info: proto=17 proto_state=01 duration=7 expire=174 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=TYOTO-ULNHQ/ vlan_cos=0/255
state=log may_dirty f00
statistic(bytes/packets/allow_err): org=415/7/1 reply=438/8/1 tuples=2
tx speed(Bps/kbps): 55/0 rx speed(Bps/kbps): 58/0
orgin->sink: org pre->post, reply pre->post dev=15->17/17->15 gwy=192.168.20.253/192.168.0.28
hook=pre dir=org act=noop 192.168.0.28:4569->192.168.20.253:4569(0.0.0.0:0)
hook=post dir=reply act=noop 192.168.20.253:4569->192.168.0.28:4569(0.0.0.0:0)
misc=0 policy_id=2 auth_info=0 chk_client_info=0 vd=0
serial=000005db tos=ff/ff app_list=0 app=0 url_cat=0
dd_type=0 dd_mode=0
total session 1

[/size]

Debug messages:

[size="2"]

2017-04-24 18:35:15 id=20085 trace_id=138 func=print_pkt_detail line=5319 msg="vd-root received a packet(proto=17, 192.168.0.28:4569->192.168.20.253:4569) from internal. "
2017-04-24 18:35:15 id=20085 trace_id=138 func=resolve_ip_tuple_fast line=5394 msg="Find an existing session, id-000005db, original direction"
2017-04-24 18:35:15 id=20085 trace_id=138 func=ipsecdev_hard_start_xmit line=178 msg="enter IPsec interface-TYOTO-ULNHQ"
2017-04-24 18:35:15 id=20085 trace_id=138 func=esp_output4 line=888 msg="IPsec encrypt/auth"
2017-04-24 18:35:15 id=20085 trace_id=138 func=ipsec_output_finish line=522 msg="send to gatewayIP via intf-ppp1"
2017-04-24 18:35:15 id=20085 trace_id=139 func=print_pkt_detail line=5319 msg="vd-root received a packet(proto=17, 192.168.20.253:4569->192.168.0.28:4569) from TYOTO-ULNHQ. "
2017-04-24 18:35:15 id=20085 trace_id=139 func=resolve_ip_tuple_fast line=5394 msg="Find an existing session, id-000005db, reply direction"
2017-04-24 18:35:15 id=20085 trace_id=139 func=__if_queue_push_xmit line=363 msg="send out via dev-lan, dst-mac-aa:bb:cc:aa:ff:f7"
2017-04-24 18:35:15 id=20085 trace_id=140 func=print_pkt_detail line=5319 msg="vd-root received a packet(proto=17, 192.168.0.28:4569->192.168.20.253:4569) from internal. "
2017-04-24 18:35:15 id=20085 trace_id=140 func=resolve_ip_tuple_fast line=5394 msg="Find an existing session, id-000005db, original direction"
2017-04-24 18:35:15 id=20085 trace_id=140 func=ipsecdev_hard_start_xmit line=178 msg="enter IPsec interface-TYOTO-ULNHQ"
2017-04-24 18:35:15 id=20085 trace_id=140 func=esp_output4 line=888 msg="IPsec encrypt/auth"
2017-04-24 18:35:15 id=20085 trace_id=140 func=ipsec_output_finish line=522 msg="send to gatewayIP  via intf-ppp1"

Only following features turned on (via GUI):[/size]

[size="2"][Basic Features][/size]

Switch Controller

VPN

WiFi Controller

[size="2"][Security Features][/size]

-None-

[size="2"][Additional Features][/size]

DNS Database

Implicit Firewall Policies

I turned off the SIP-Helper and SIP-NAT-Tracer. I set default-voip-alg-mode to kernel-helper-based.

[size="2"]

config system settings
    set sip-helper disable
    set sip-nat-trace disable
    set default-voip-alg-mode kernel-helper-based
end

[/size]

Also deleted SIP entry from config system session-helper.

Disabled RTP from VoIP default profile.

Still no use.

Please help me.

[AtiT · Accepted solution]

Hello,

the lines:

2017-04-24 18:35:15 id=20085 trace_id=140 func=ipsecdev_hard_start_xmit line=178 msg="enter IPsec interface-TYOTO-ULNHQ"
2017-04-24 18:35:15 id=20085 trace_id=140 func=esp_output4 line=888 msg="IPsec encrypt/auth"

inticates to me that the traffic is sent to the IPSec tunnel.

According to your description "the servers starts talking" seems to me that it is working?

Can you confirm that?

The reason why it is working after the session clear should be that the tunnel was down when the servers wanted to communicate.

It is always good to set a blackhoule route to the IPSec subnet in case the tunnel goes down the sessions will be not created and idle in the session list.

[AtiT · Accepted solution]

Hello,

the lines:

2017-04-24 18:35:15 id=20085 trace_id=140 func=ipsecdev_hard_start_xmit line=178 msg="enter IPsec interface-TYOTO-ULNHQ"
2017-04-24 18:35:15 id=20085 trace_id=140 func=esp_output4 line=888 msg="IPsec encrypt/auth"

inticates to me that the traffic is sent to the IPSec tunnel.

According to your description "the servers starts talking" seems to me that it is working?

Can you confirm that?

The reason why it is working after the session clear should be that the tunnel was down when the servers wanted to communicate.

It is always good to set a blackhoule route to the IPSec subnet in case the tunnel goes down the sessions will be not created and idle in the session list.

[batbayar1001]

Hi,

Thank you for the reply :)

I believe the tunnel was up even before the diag sys session clear command, because I could SSH both servers, run tcpdump on both of them, servers could ping each other.

Only IAX2 traffic couldn't pass through the tunnel. After I run the diag sys session clear command, I can see in the tcpdump log that servers can talk each other on IAX2 (udp 4569) port, and I can make/transfer voice calls between the servers.

But when I restart the firewall, problem will come back and it starts to send IAX2 packets to its default gateway.

[JonathanTorian_FTNT · Accepted solution]

Do you happen to have a blackhole static route sent for your network going through the VPN?  Essentially what you may be observing is that when the VPN goes down, the route gets sticky on the default route.  To fix this, create a static blackhole route with a higher priority than the IPSec route.

That should take care of the issue in the future.

[batbayar1001]

You guys are awesome!

Adding static blackhole route fixed the strange behavior.

I was almost thinking that in a magic world of FortiEarth, a FortiGandalf yelling at my packets "YOU SHALL NOT PASS!!!" on the Durin's IPSec Bridge over Internet-dum.  But nope, there was a technical solution 

Thank you,

Bat