Technical Tip: FGSP Configuration Guide for Session Sync and Config Sync

pywong · ‎09-18-2019

Description

This article provides CLI configuration guidelines for Session Sync and Config Sync in Fortigate FGSP (FortiGate Session Life Support Protocol) setup.
FGSP parameters are configured from the CLI only.

Scope

FortiGate.

Solution

Topology:

                 FGT-A
           [port2]   [port3]
              |         |
              |         |
              |         |
           [port2]   [port3]
                 FGT-B

FGT-A port2 IP address : 10.47.1.124
FGT-B port2 IP address : 10.47.1.150
Port2 is in the root VDOM, used for peering.
Port3 is used as a heartbeat interface.

Configure an FGSP HA cluster-sync instance:

FGT-A:

config system cluster-sync
    edit 1
            set peerip 10.47.1.150
    set peervd "root"
            set syncvd "vd1"
        next
end

FGT-B:

config system cluster-sync
    edit 1
            set peerip 10.47.1.124
    set peervd "root"
            set syncvd "vd1"
        next
end

Notes:

'peerip' is the IP address of an interface of another FortiGate in the FGSP cluster that this configuration synchronizes sessions to.
'peervd' is the name of the virtual domain that contains the session synchronization link interface on the peer unit.
Usually, both peers would have the same peervd. Multiple session synchronization configurations can use the same peervd. The default VDOM name is root.
'syncvd' is the name of one or more VDOMs that should be synchronized by this cluster-sync instance. If multiple VDOMs are not enabled, syncvd should be set to root, which is the default setting.

Configure Session Synchronization:

Synchronize NAT sessions:

config system ha
set session-pickup enable
set session-pickup-nat enable

Synchronizes UDP and ICMP sessions:

config system ha
set session-pickup enable
set session-pickup-connectionless enable

Synchronizes exception sessions also called asymmetric sessions:

config system ha
set session-pickup enable
set session-pickup-expectation enable

Enable Configuration Synchronization:

FGT-A:

config system ha
    set group-id 79
    set group-name "jwfgsp"
    set hbdev "port3" 50
    set standalone-config-sync enable
    set priority 200
end

FGT-B:

config system ha
    set group-id 79
    set group-name "jwfgsp"
    set hbdev "port3" 50
    set standalone-config-sync enable
    set priority 100
end

Useful diagnostic commands:

diagnose sys session sync
sync_ctx: sync_started=1, sync_tcp=1, sync_others=0,
sync_expectation=0, sync_redir=0, sync_nat=1, stdalone_sesync=1.
sync: create=243:0, update=1043, delete=0:0, query=0
recv: create=0:0, update=0, delete=0:0, query=0
ses pkts: send=0, alloc_fail=0, recv=0, recv_err=0 sz_err=0
udp pkts: send=1220, recv=0
nCfg_sess_sync_num=5, mtu=1500
sync_filter:
1: vd=1, szone=0, dzone=0, saddr=0.0.0.0:0.0.0.0, daddr=0.0.0.0:0.0.0.0, sport=0-65535, dport=0:65535

diagnose sys session list

The session state on the FortiGate where the session is first created will show as 'synced'. For the same session which is sync'ed over to the peer FortiGate, it will have the session state 'syn_ses'. Use grep to count the number of sessions.

#FGT-A# diagnose sys session list
session info: proto=6 proto_state=01 duration=4 expire=3595 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=may_dirty synced
statistic(bytes/packets/allow_err): org=3787/13/1 reply=830/9/1 tuples=2
tx speed(Bps/kbps): 800/6 rx speed(Bps/kbps): 175/1
orgin->sink: org pre->post, reply pre->post dev=5->18/18->5 gwy=192.168.100.1/10.173.1.234
hook=post dir=org act=snat 10.173.1.234:52403->151.101.2.49:443(192.168.100.2:52403)
hook=pre dir=reply act=dnat 151.101.2.49:443->192.168.100.2:52403(10.173.1.234:52403)
pos/(before,after) 0/(0,0), 0/(0,0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=1
serial=0000064a tos=ff/ff app_list=0 app=0 url_cat=0
rpdb_link_id = 00000000
dd_type=0 dd_mode=0

#FGT-A# diagnose sys session list | grep synced -c

#FGT-B# diagnose sys session list
session info: proto=6 proto_state=01 duration=8 expire=3591 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=may_dirty syn_ses
statistic(bytes/packets/allow_err): org=0/0/0 reply=0/0/0 tuples=2
tx speed(Bps/kbps): 0/0 rx speed(Bps/kbps): 0/0
orgin->sink: org pre->post, reply pre->post dev=5->18/18->5 gwy=0.0.0.0/0.0.0.0
hook=post dir=org act=snat 10.173.1.234:52403->151.101.2.49:443(192.168.100.2:52403)
hook=pre dir=reply act=dnat 151.101.2.49:443->192.168.100.2:52403(10.173.1.234:52403)
pos/(before,after) 0/(0,0), 0/(0,0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=1
serial=0000064a tos=ff/ff app_list=0 app=0 url_cat=0
rpdb_link_id = 00000000
dd_type=0 dd_mode=0

#FGT-B# diagnose sys session list | grep syn_ses -c

Check HA status:

get sys ha status

FGT-A (global) # get sys ha status
HA Health Status: OK
Model: FortiGate-VM64-KVM
Mode: ConfigSync
Group: 79
Debug: 0
Cluster Uptime: 0 days 3:29:41
Cluster state change time: 2019-05-14 10:32:05
Master selected using:
    <2019/05/14 10:32:05> FGVM0100001XXXX9 is selected as the master because it has the largest value of override priority.
    <2019/05/14 10:31:12> FGVM0100001XXXX9 is selected as the master because it's the only member in the cluster.
    <2019/05/14 10:24:30> FGVM0100001XXXX9 is selected as the master because it's the only member in the cluster.
ses_pickup: enable, ses_pickup_delay=disable
override: disable
Configuration Status:
    FGVM0100001XXXX9(updated 3 seconds ago): in-sync
    FGVM0100001XXXX8(updated 1 seconds ago): in-sync
System Usage stats:
    FGVM0100001XXXX9(updated 3 seconds ago):
        sessions=7, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=38%
    FGVM0100001XXXX8(updated 1 seconds ago):
        sessions=5, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=38%
HBDEV stats:
    FGVM0100001XXXX9(updated 3 seconds ago):
        port3: physical/10000full, up, rx-bytes/packets/dropped/errors=22028298/76225/0/0, tx=36224902/77755/0/0
    FGVM0100001XXXX8(updated 1 seconds ago):
        port3: physical/10000full, up, rx-bytes/packets/dropped/errors=26021989/70773/0/0, tx=20856052/67732/0/0
Master: FGT-A        , FGVM0100001XXXX9, cluster index = 1
Slave : FGT-B        , FGVM0100001XXXX8, cluster index = 0
number of vcluster: 1
vcluster 1: work 169.254.0.2
Master: FGVM0100001XXXX9, operating cluster index = 0
Slave : FGVM0100001XXXX8, operating cluster index = 1

#FGT-B (global) # get sys ha status
HA Health Status: OK
Model: FortiGate-VM64-KVM
Mode: ConfigSync
Group: 79
Debug: 0
Cluster Uptime: 0 days 3:30:39
Cluster state change time: 2019-05-14 10:32:05
Master selected using:
    <2019/05/14 10:32:05> FGVM0100001XXXX9 is selected as the master because it has the largest value of override priority.
    <2019/05/14 10:31:25> FGVM0100001XXXX8 is selected as the master because it's the only member in the cluster.
    <2019/05/14 10:24:23> FGVM0100001XXXX8 is selected as the master because it's the only member in the cluster.
ses_pickup: enable, ses_pickup_delay=disable
override: disable
Configuration Status:
    FGVM0100001XXXX8(updated 3 seconds ago): in-sync
    FGVM0100001XXXX9(updated 5 seconds ago): in-sync
System Usage stats:
    FGVM0100001XXXX8(updated 3 seconds ago):
        sessions=9, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=38%
    FGVM0100001XXXX9(updated 5 seconds ago):
        sessions=8, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=38%
HBDEV stats:
    FGVM0100001XXXX8(updated 3 seconds ago):
        port3: physical/10000full, up, rx-bytes/packets/dropped/errors=26149775/71119/0/0, tx=20958694/68065/0/0
    FGVM0100001XXXX9(updated 5 seconds ago):
        port3: physical/10000full, up, rx-bytes/packets/dropped/errors=22131444/76570/0/0, tx=36351856/78088/0/0
Slave : FGT-B        , FGVM0100001XXXX8, cluster index = 0
Master: FGT-A        , FGVM0100001XXXX9, cluster index = 1
number of vcluster: 1
vcluster 1: work 169.254.0.2
Slave : FGVM0100001XXXX8, operating cluster index = 1
Master: FGVM0100001XXXX9, operating cluster index = 0

Check Config Synchronization:

diagnose sys ha checksum cluster

#FGT-A (global) # diagnose sys ha checksum cluster

================== FGVM0100001XXXX9 ==================

is_manage_master()=1, is_root_master()=1
debugzone
global: 3f fd 66 00 3a ba 0b 09 ae 5b 41 4f 9c f0 81 01
vd1: c4 6e 79 0c 82 12 3e 39 d2 b0 f4 df 5e 30 32 aa
root: 17 c9 6c 54 1f 9f a1 0d 24 60 a8 cb cb 48 99 02
all: 2c 26 c9 5c 53 5c 15 91 5d ef 76 81 42 80 6a 84

checksum
global: 3f fd 66 00 3a ba 0b 09 ae 5b 41 4f 9c f0 81 01
vd1: c4 6e 79 0c 82 12 3e 39 d2 b0 f4 df 5e 30 32 aa
root: 17 c9 6c 54 1f 9f a1 0d 24 60 a8 cb cb 48 99 02
all: 2c 26 c9 5c 53 5c 15 91 5d ef 76 81 42 80 6a 84

================== FGVM0100001XXXX8 ==================

is_manage_master()=0, is_root_master()=1
debugzone
global: 3f fd 66 00 3a ba 0b 09 ae 5b 41 4f 9c f0 81 01
vd1: c4 6e 79 0c 82 12 3e 39 d2 b0 f4 df 5e 30 32 aa
root: 17 c9 6c 54 1f 9f a1 0d 24 60 a8 cb cb 48 99 02
all: 2c 26 c9 5c 53 5c 15 91 5d ef 76 81 42 80 6a 84

checksum
global: 3f fd 66 00 3a ba 0b 09 ae 5b 41 4f 9c f0 81 01
vd1: c4 6e 79 0c 82 12 3e 39 d2 b0 f4 df 5e 30 32 aa
root: 17 c9 6c 54 1f 9f a1 0d 24 60 a8 cb cb 48 99 02
all: 2c 26 c9 5c 53 5c 15 91 5d ef 76 81 42 80 6a 84

#FGT-B (global) # diagnose sys ha checksum cluster

================== FGVM0100001XXXX8 ==================

is_manage_master()=0, is_root_master()=1
debugzone
global: 3f fd 66 00 3a ba 0b 09 ae 5b 41 4f 9c f0 81 01
vd1: c4 6e 79 0c 82 12 3e 39 d2 b0 f4 df 5e 30 32 aa
root: 17 c9 6c 54 1f 9f a1 0d 24 60 a8 cb cb 48 99 02
all: 2c 26 c9 5c 53 5c 15 91 5d ef 76 81 42 80 6a 84

checksum
global: 3f fd 66 00 3a ba 0b 09 ae 5b 41 4f 9c f0 81 01
vd1: c4 6e 79 0c 82 12 3e 39 d2 b0 f4 df 5e 30 32 aa
root: 17 c9 6c 54 1f 9f a1 0d 24 60 a8 cb cb 48 99 02
all: 2c 26 c9 5c 53 5c 15 91 5d ef 76 81 42 80 6a 84

================== FGVM0100001XXXX9 ==================

is_manage_master()=1, is_root_master()=1
debugzone
global: 3f fd 66 00 3a ba 0b 09 ae 5b 41 4f 9c f0 81 01
vd1: c4 6e 79 0c 82 12 3e 39 d2 b0 f4 df 5e 30 32 aa
root: 17 c9 6c 54 1f 9f a1 0d 24 60 a8 cb cb 48 99 02
all: 2c 26 c9 5c 53 5c 15 91 5d ef 76 81 42 80 6a 84

checksum
global: 3f fd 66 00 3a ba 0b 09 ae 5b 41 4f 9c f0 81 01
vd1: c4 6e 79 0c 82 12 3e 39 d2 b0 f4 df 5e 30 32 aa
root: 17 c9 6c 54 1f 9f a1 0d 24 60 a8 cb cb 48 99 02
all: 2c 26 c9 5c 53 5c 15 91 5d ef 76 81 42 80 6a 84

For creating a FGSP setup with two FortiGate HA clusters, where should be synchronized the sessions between the clusters, should be performed below configurations:

FGT-A-Primary and FGT-A-Secondary:

config system standalone-cluster
set standalone-group-id 1

set group-member-id 0
    config cluster-peer
    edit 1
    set peerip 10.102.1.64
    next
end

FGT-B-Primary and FGT-B-Secondary:

config system standalone-cluster
set standalone-group-id 1

set group-member-id 1
    config cluster-peer
    edit 1
    set peerip 10.102.1.39
    next
end

The 'set standalone-group-id' should be the same for the FortiGate HA clusters, meanwhile 'group-member-id' should be the same for the members of an HA cluster but different for FortiGate Firewalls outside the HA cluster.

Related articles:

Configuration Guide: FortiGate Session Life Support Protocol (FGSP).

Technical Note: FGSP configuration notes.