OT segregation with intervlan traffic

[jfrye]

I am trying to use a fortigate 60F to segregate my OT network, so SCADA from ICS.  I already have switches in place, so trying to use what is there.  Currently testing on the bench, with 2 managed switches connected to the FGT, each switch represents one VLAN, as below:

VLAN 10 - 10.1.0.0/24

VLAN 11 - 10.1.1.0/24

I have assigned physical interfaces (left at 0.0.0.0/0) to VLAN switches (as above). 

Alternatively assigned VLAN interfaces to the physical interfaces (0.0.0.0/0 again).

Both with bidirectional firewall policies.

Using VLAN switches, which was the best result, I can link to one switch, and get to the FGT but not to a device or other switch in the other VLAN; the FGT can ping devices on either VLAN.  What am I missing?  Thanks.

[AEK]

Which interfaces did you put in ICS Zone and SCADA Zone? Can you share a screenshot of the those interfaces' config?

On the other hand (but not related to your issue) I don't think it is a good idea to NAT the traffic, unless you have a good reason.

[jfrye]

I put the physical interfaces, internal3 and internal5, into ICS and SCADA zones, respectively.  Photos attached.  Should the subnet be done differently?  My switches are connected by access ports, so vlan untagged, would that matter or would it go off of the subnetting for the policy?

NAT was on by default...  Thanks.

[AEK]

Did you check if your devices havethe respective FG interface IP addresses as their gateways?

[jfrye]

Yes they do, using the 10.1.0.145 and 10.1.1.145 as their gateways.

[AEK]

Can you run the following while you do the ping test?

diag debug flow filter  (filter by IP and/or ICMP)
diag debug console timestamp enable
diag debug flow show iprope enable

diag debug flow show function-name enable
diag debug flow trace start 100
diag debug enable

[jfrye]

This is my result, pinging from GUI of switch to switch.  I had no luck trying from my machine connected to a switch.

[AEK]

According to the shared logs it seems traffic is allowed.

On the other hand, I'm not used to VLAN switches and can't tell if your configuration is supposed to work as expected.

So is there any reason for which you are using VLAN switch instead of software switch or hardware switch?

[jfrye]

Using VLAN switches because that is the closest I got to working properly, i.e. could at least ping the FGT from either switch.  My network is VLAN-ed, so I figured the FGT would need to know that.

It looked like the return packet didn't get sent, no reason why though.  See attached.

[jfrye]

Assigning an IP to both physical interfaces, then assigning them to zones, and making the policies zone based results in this same debug.

Hardware switch results in the same debug.

Software switch did not work.

It looks like the packets get to the destination IP, but does not make it back.