Skip to main content
mmkor
mmkor
New Member
October 19, 2022
Question

Fortilink L3

  • October 19, 2022
  • 5 replies
  • 4561 views

Hello,

 

situation:

2x FG200F connected to a HP Comware switch cluster via LAG (2x2x10Gbit)

Bought 13x Fortiswitch 148F as access switches

 

Default VLAN on the trunk is the legacy network and won't be easily changed
Several VLANs on that trunk working as intended, fortigate is primary router for legacy network and several DMZs

Spanning VLANs via L2 Fortilink is not needed, just Centralized Management and eventually NAC configuration

 

Original plan was to create a management VLAN (3) and have the Fortiswitches managed from the Fortigate UI, mostly based on Fortilink over TCP – InfoSec Monkey.

On the FG200F Fortilink is configured to an unused interface (10.255.3.1).

Policy allowing VLAN3 to fortilink interface is active.

The interface facing the fortiswitch uplink (port49) has PVID 3.

 

The switch can ping the IP of the fortigate on VLAN3 (10.100.3.1), and also the fortilink interface through that network as a gateway. I can see the CAPWAP attempts on the fortigate interface capture on VLAN3, but the fortigate doesn't respond to that, only for the NTP.

The interface facing the fortiswitch has PVID 3

 

What am I missing? I am considering using Fortilink over VXLAN from the official documentation to circumvent this issue, but this seems like another extra layer of complexity which I would like to avoid, as adding extra cables to the core switches

 

 

Kind regards,

Michael

 

5 replies

distillednetwork
distillednetwork
Explorer II
October 23, 2022

Are the switches getting DHCP or a static IP?  If DHCP make sure you have dhcp-option-code 138 or if its static set the ac-discovery.

 

Also, did you do "set fortlink-l3-mode enable" on the fortiswitch port connected to the HP

mmkor
mmkorAuthor
New Member
October 24, 2022

Hello distillednetwork,

With firmware 7.02 which came with the switch I tried both dhcp option field 138 as well as 

 

config switch-controller global
set ac-discovery-type static
  config ac-list
      edit 1
         set ipv4-address 10.255.3.1
       next
   end
end

 

After using set fortlink-l3-mode enable on the uplink interface I see CAPWAP attempts arriving at 10.255.3.1 on the fortigate

 

After upgrading to 7.22 that command isn't available anymore, it was been deprecated in 7.21. as per (FortiLink Guide | FortiSwitch 7.2.1 | Fortinet Documentation Library) static isl link is supposed to be used, whatever it is supposed to do.

 

 

 

On the fortigate with

diag debug application cw_acd -1

diag debug enable

 

I see about 2 times per minute that the switch sends something to the controller (Hexdump contains FS148) but the devices don't seem to handshake

 

Kind regards,

Michael

distillednetwork
distillednetwork
Explorer II
October 24, 2022

When you say you have an unused interface for fortilink, is this an unused port?  Is the port in an up state?

 

What if you run:
diag debug flow filter addr <fortiswitch ip>

diag debug en

diag debug flow trace start 25

Can you see bidirectional traffic?  Are there any drops reported?

Would you mind sharing the configurations and a diagram to better understand?  

mmkor
mmkorAuthor
New Member
October 24, 2022

Hello distillednetwork

 

Only incoming packages visible, no drops but no responses aswell unless I ping from the switch:

 

id=65308 trace_id=72 func=print_pkt_detail line=5892 msg="vd-root:0 received a packet(proto=17, 10.100.3.101:5246->10.255.3.1:5246) tun_id=0.0.0.0 from V003. "
id=65308 trace_id=72 func=resolve_ip_tuple_fast line=5980 msg="Find an existing session, id-0161553a, original direction"

 

When switch is DHCP client, it gets 10.100.3.101, on manual it gets 10.100.3.11

 

forti1.png

For testing puposes active Policies:

 

allow all VLAN 1, VLAN 3 into Fortilink

allow all Fortilink into VLAN1, VLAN3

 

I have seen in a published Youtube video from Fortinet Taiwan that connecting Fortilink L3 to a LACP Fortilink with all members down works

 

Kind regards,

Michael

 

 

distillednetwork
distillednetwork
Explorer II
October 24, 2022

the native vlan is 3 from the Core switchstack to the FS148F, but do you still have the default vlan on the FortiLink as 4094?  Can you show the configuration of the Fortiswtich?  You can have a different vlan on the FS148F because it is native on both sides.  

mmkor
mmkorAuthor
New Member
October 25, 2022

Hello distillednetwork,

 

When I set the default vlan on the switchtrunk to 4094, things get dead silent on the connection to the fortigate, so i unset it again. I think vlan4094 is more relevant in 7.21 which has the "set fortilink-l3-mode enable" option.

 

Switch config (static ip):

 

Spoiler

config system interface
edit "internal"
set mode static
set ip 10.100.3.11 255.255.255.0
set allowaccess ping https http ssh
next
end

config router static
edit 1
set device "internal"
set dst 0.0.0.0 0.0.0.0
set gateway 10.100.3.1
set status enable
next
end

config system global
# set auto-isl enable
set dst enable
# set hostname "S148FNTF22001912"
set timezone 26
end

config system ntp
set allow-unsync-source enable
config ntpserver
edit 1
set server "10.100.3.1"
next
end
set source-ip 10.100.3.11
set ntpsync enable
end

config switch-controller global
set ac-discovery-type static
config ac-list
edit 1
set ipv4-address 10.255.3.1
next
end
end


config switch trunk
edit "__FoRtILnk0L3__"
set auto-isl 1
set static-isl enable
set static-isl-auto-vlan disable
set members "port49"
next

end

 

 

More important seems to me that the fortilink service on the fortigate registers the attempts, but doesn't react as intended.

"diagnose debug application cu_acd -1" on the fortigate gives me

 

 

Spoiler

33652.170 cuEventPollLoop: ***********************1 num 1**********************
33652.170 ==========================cuAcProcRawMsg 13 1=========================
33652.170 DAEMON HEX DUMP (0x4b6ec58, 1327) at (1205,cuAcProcRawMsg)
33652.170 cuBufAlloc: get buffer 0x8cedd80 (0x8cedd70) from free list (573/576/4160).
33652.170 CAPWAP Hdr: P/T=0/0 len=2 RID=0 WBID=1 T=0 F=0 L=0 W=0 M=0 K=0 resv=0 frag=0/0 resv=0
33652.170 CAPWAP Control Header Dump:
33652.171 msgType : 1 DISCOVERY_REQ
33652.171 seqNum : 73
33652.171 msgElemLen : 1314
33652.171 flags : 0
33652.171 cu_me_decoder_vsp: me type 37 len 8 vid: 12356 eid: 1
33652.171 cu_me_decoder_general: me type 20 len 1
33652.171 cu_me_decoder_wtp_board_data: me type 38 len 60
33652.171 cu_me_decoder_wtp_board_data: me type 38 len 0 vId 12356 model 0x8cee6a8 serial 0x8cee6b0
33652.171 cu_me_decoder_wtp_desc: me type 39 len 117
33652.171 cu_me_decoder_wtp_desc: me type 39 len 0 radio 1/1 hw 0x8cee730 asw 0x8cee738 boot 0x8cee760
33652.171 cu_me_decoder_wtp_frame_tunnel_mode: me type 41 len 1
33652.171 cu_me_decoder_wtp_frame_tunnel_mode: me type 41 len 1 wtp_tunnel_type 0x0
33652.171 cu_me_decoder_wtp_mac_type: me type 44 len 1
33652.171 cu_me_decoder_wtp_mac_type: me type 44 len 1 mac_type 0
33652.171 cu_me_decoder_general: me type 56 len 1
33652.171 cu_me_decoder_version: me type 57 len 1
33652.171 cu_me_decoder_version: me type 57 len 1 version:1
33652.171 cu_me_decoder_fl_attr: me type 58 len 11
33652.171 caplen 2 mecaplen:2 remain=0 cap0=0x94c3f9d7 cap1=0x22675
33652.171 cu_me_decoder_fl_attr: me type 58 len 11 blk:2
33652.171 Added switch (0, FORTISWITCH_SN) to dynamically learnt tree
33652.171 start: ADDED sw=FORTISWITCH_SN
33652.171 start: ver=1 seq=1 0 FORTISWITCH_SN
33652.171 switch: 52 0 3 8
33652.171 FP(0 FORTISWITCH_SN): 284
<model type="s148fn">
<portgroup id="portgroup1" label="" rows="2">
<port id="port" start="1" switch_name="fsw" num="48"/>
</portgroup>
<portgroup id="portgroup2" label="SFP+" rows="2">
<port id="port" start="49" switch_name="fsw" type="fiber" num="4"/>
</portgroup>
</model>


33652.171 port(0, FORTISWITCH_SN): 5 port1 0 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 5 port2 1 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 5 port3 2 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 5 port4 3 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 5 port5 4 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 5 port6 5 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 5 port7 6 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 5 port8 7 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 5 port9 8 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port10 9 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port11 10 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port12 11 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port13 12 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port14 13 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port15 14 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port16 15 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port17 16 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port18 17 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port19 18 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port20 19 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port21 20 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port22 21 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port23 22 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port24 23 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port25 24 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port26 25 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port27 26 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port28 27 64 0xcf
33652.171 port(0, FORTISWITCH_SN): 6 port29 28 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port30 29 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port31 30 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port32 31 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port33 32 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port34 33 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port35 34 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port36 35 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port37 36 64 0xcf
33652.172 cuAcProcDiscoverReqMsg: received DISCOVERY_REQ from FORTISWITCH_SN 10.100.3.11/5246
33652.172 cuAcProcDiscoverReqMsg Interface V003 is not authorized for FORTISWITCH_SN
33652.172 cuBufFree: free buffer 0x8cedd80 (0x8cedd70) to free list (574/576/4160).
33652.172 DAEMON HEX DUMP (0x4b6ec58, 579) at (1205,cuAcProcRawMsg)
33652.172 cuBufAlloc: get buffer 0x8cedd80 (0x8cedd70) from free list (573/576/4160).
33652.172 CAPWAP Hdr: P/T=0/0 len=2 RID=0 WBID=1 T=0 F=0 L=0 W=0 M=0 K=0 resv=0 frag=0/0 resv=0
33652.172 CAPWAP Control Header Dump:
33652.172 msgType : 1 DISCOVERY_REQ
33652.172 seqNum : 74
33652.172 msgElemLen : 566
33652.172 flags : 0
33652.172 cu_me_decoder_vsp: me type 37 len 8 vid: 12356 eid: 1
33652.172 cu_me_decoder_general: me type 20 len 1
33652.172 cu_me_decoder_wtp_board_data: me type 38 len 60
33652.172 cu_me_decoder_wtp_board_data: me type 38 len 0 vId 12356 model 0x8cee6a8 serial 0x8cee6b0
33652.172 cu_me_decoder_wtp_desc: me type 39 len 117
33652.172 cu_me_decoder_wtp_desc: me type 39 len 0 radio 1/1 hw 0x8cee730 asw 0x8cee738 boot 0x8cee760
33652.172 cu_me_decoder_wtp_frame_tunnel_mode: me type 41 len 1
33652.172 cu_me_decoder_wtp_frame_tunnel_mode: me type 41 len 1 wtp_tunnel_type 0x0
33652.172 cu_me_decoder_wtp_mac_type: me type 44 len 1
33652.172 cu_me_decoder_wtp_mac_type: me type 44 len 1 mac_type 0
33652.172 cu_me_decoder_general: me type 56 len 1
33652.172 cu_me_decoder_version: me type 57 len 1
33652.172 cu_me_decoder_version: me type 57 len 1 version:1
33652.172 cu_me_decoder_fl_attr: me type 58 len 11
33652.172 caplen 2 mecaplen:2 remain=0 cap0=0x94c3f9d7 cap1=0x22675
33652.172 cu_me_decoder_fl_attr: me type 58 len 11 blk:2
33652.172 mid: ver=1 seq=2 0 FORTISWITCH_SN
33652.172 port(0, FORTISWITCH_SN): 6 port38 37 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port39 38 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port40 39 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port41 40 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port42 41 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port43 42 64 0xcf
33652.172 port(0, FORTISWITCH_SN): 6 port44 43 64 0xcf
33652.173 port(0, FORTISWITCH_SN): 6 port45 44 64 0xcf
33652.173 port(0, FORTISWITCH_SN): 6 port46 45 64 0xcf
33652.173 port(0, FORTISWITCH_SN): 6 port47 46 64 0xcf
33652.173 port(0, FORTISWITCH_SN): 6 port48 47 64 0xcf
33652.173 port(0, FORTISWITCH_SN): 6 port49 48 1024 0x604b0
33652.173 port(0, FORTISWITCH_SN): 6 port50 49 1024 0x604b0
33652.173 port(0, FORTISWITCH_SN): 6 port51 50 1024 0x604b0
33652.173 port(0, FORTISWITCH_SN): 6 port52 51 1024 0x604b0
33652.173 received all for 0 FORTISWITCH_SN
33652.173 End: seq=0 type=68 0 FORTISWITCH_SN
33652.173 cuAcProcDiscoverReqMsg: received DISCOVERY_REQ from FORTISWITCH_SN 10.100.3.11/5246
33652.173 cuAcProcDiscoverReqMsg Interface V003 is not authorized for FORTISWITCH_SN
33652.173 cuBufFree: free buffer 0x8cedd80 (0x8cedd70) to free list (574/576/4160).
33652.173 ==========================cuAcProcRawMsg 13 2=========================
33652.173 cuEventPollLoop: ***********************2 num 1**********************

 

I think I am missing a small setting on the fortigate

Kind regards,

Michael

Terms & ConditionsAccessibility statement