No seperation using VRFs

[Immu]

Hi,

I have an issue with use of VRFs.
I configured the FGT like descriped in the following instruction: https://docs.fortinet.com/document/fortigate/7.2.3/administration-guide/752950

The FGT has two IPsec tunnels, one for primary and one as backup.
All traffic has to go through the tunnels, which means there are two default routes with exit interfaces of the VPNs.
If I don't use VRF routing, I have to route each specific IP addresse or range e.g. IP of the peer, LDAP or TACACS+ servers and so on via WAN interfaces...
So I configured a VRF for the customer intranet, which contains tunnel interfaces and LAN.
The other VRF is for the two WAN interfaces primary and secondary.

At the end I have multiple (default) routes:
- two default routes through the tunnels in VRF1 (different AD)
- two default routes through the physical WAN interfaces in default VRF (different AD)
- two blackhole routes for each VRF

config router static
edit 1
set device "VPN-RZ"
set comment "VRF=1"
next
edit 2
set distance 200
set priority 11
set device "VPN-RZ-Backup"
set comment "VRF=1"
next
edit 3
set gateway 100.70.1.1
set device "wan1"
set comment "VRF=0"
next
edit 4
set distance 200
set priority 11
set device "cfg" #cfg is the secondary interface. Don't ask me, why I named it like that :p
set comment "VRF=0"
set dynamic-gateway enable
next
edit 5
set distance 254
set comment "VRF=1"
set blackhole enable
set vrf 1
next
edit 6
set distance 254
set comment "VRF=0"
set blackhole enable
next
end

It seemes to work at the first look... the tunnels get up and all traffic from or for the LAN goes through them.
But I can reach the IP addresses from LAN to WAN via ICMP and reverse.
I would understand, if the traffic would go through multiple hops.
But the FGT routes the traffic locally?!
So there is just one hop, which shouldn't be possible like descriped in the manual above!

So I thought - ok maybe it's a bug and an update of firmware would be helpful. So I did, twice.
And the issue still exists!

The FortiGate shows two routing tables, but the traffic seems not to be seperated.

Here is the interface configuration:
config system interface
edit "wan"
set vdom "root"
set ip <ip address>
set allowaccess ping https ssh snmp http fgfm
set type physical
set lldp-reception enable
set role wan
set snmp-index 1
next
edit "lan1"
set vdom "root"
set type physical
set snmp-index 11
next
edit "lan2"
set vdom "root"
set type physical
set snmp-index 9
next
edit "lan3"
set vdom "root"
set vrf 1
set ip 10.139.240.1 255.255.255.0
set allowaccess ping https ssh snmp http
set type physical
set device-identification enable
set lldp-reception enable
set lldp-transmission enable
set role lan
set snmp-index 8
next
edit "a"
set vdom "root"
set mode dhcp
set allowaccess ping https ssh snmp http
set type physical
set alias "MF-Backup"
set lldp-reception enable
set role wan
set snmp-index 2
next
edit "lan"
set vdom "root"
set vrf 1
set ip 10.139.239.1 255.255.255.0
set allowaccess ping https ssh http fgfm fabric
set type hard-switch
set alias "LAN"
set stp enable
set device-identification enable
set lldp-reception enable
set lldp-transmission enable
set role lan
set snmp-index 5
next
edit "cfg"
set vdom "root"
set mode pppoe
set allowaccess ping https ssh http
set snmp-index 7
set username <username>
set password <password>
set interface "a"
set vlanid 7
next
edit "VPN-RZ"
set vdom "root"
set vrf 1
set type tunnel
set snmp-index 6
set interface "wan"
next
edit "VPN-RZ-Backup"
set vdom "root"
set vrf 1
set type tunnel
set snmp-index 10
set interface "cfg"
next
end

[AEK]

Hi Immu

Once we migrated from PAN to FGT we had to configure VRF, so we spent much time to get work properly but without success. So we had to drop VRFs and replaces it with VDOMs.

On PAN it worked like a charm but unfortunately on FGT according to my experience it is not that mature.

Hope this will help you save time and hope Fortinet will fix it soon.

[Immu]

Hi AEK,

sorry for the late reply.

I also read that, I mean that VDOMs are the better solutions instead VRFs.

Acutally VDOMs and VRFs are not the same, but if it helps...

I will check it out :)

Big thanks

[gfleming]

Can you post output of:

get router info routing-table all

Also, is there a reason you are defaulting to using VRFs here? Keep in mind this is a firewall which just so happens to also be a very high performance router and VPN device.

You can leverage the built-in nature of the FortiGate firewall capabilities to do what you want here.

Create all your connectivity, including WAN links, IPSec tunnels, lan interfaces, etc. Nothing will communicate between interfaces until you create explicit FW policies for the traffic.

IMO you should be able to do this all by just leveraging FW policies....

[Immu]

Hi Graham,

sorry for the late reply.

Routing table when both WAN interfaces are up:

# get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
V - BGP VPNv4
* - candidate default

Routing table for VRF=0
S* 0.0.0.0/0 [10/0] via 100.70.1.1, wan, [1/0]
C 100.70.1.0/24 is directly connected, wan

Routing table for VRF=1
S* 0.0.0.0/0 [10/0] via VPN-RZ tunnel 100.70.1.1, [1/0]
C 10.139.239.0/24 is directly connected, lan

Routing table for VRF=2
S* 0.0.0.0/0 [10/0] via 100.70.4.49, a, [1/0]
C 100.70.4.48/30 is directly connected, a

Routing table when primary WAN interface is down:

# get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
V - BGP VPNv4
* - candidate default

Routing table for VRF=0
S* 0.0.0.0/0 [254/0] is a summary, Null, [1/0]

Routing table for VRF=1
S* 0.0.0.0/0 [200/0] via VPN-RZ-Backup tunnel xxx.xxx.xxx.xxx, [12/0]
C 10.139.239.0/24 is directly connected, lan

Routing table for VRF=2
S* 0.0.0.0/0 [10/0] via 100.70.4.49, a, [1/0]
C 100.70.4.48/30 is directly connected, a

Internet connection is still available, while primary interface is down.

What confuses me is that IP addresses can locally communicate between VRFs.

This should only be possible, if VRF-Leaking is in use (and it's not!).

But I will try the suggestion of AEK.

Thanks :)

PS: Meanwhile I have created 2 additional VRFs, so there are three tables.

[AEK]

Some customers use VRF for security purpose.. just for extra isolation. I agree that this isolation can be done by firewall policies but sometimes some corporate policies may seem a bit strange for us.

[Immu]

(My) problem is solved... I just forgot the post...

I solved (my) problem with policy routes.

I have the default VRF only and there are just three static routes (if all paths are up):

- default route for primary path

- second default route, which gets generated automatically by DHCP addressing mode (same AD as primary route, but lower prio)

- blackhole route

And the policy routes:

- any from LAN interface to 0.0.0.0/0 via primary VPN

- any from LAN interface to 0.0.0.0/0 via secondary VPN

The exit-interface is not enough to successfully configure policy routes... a next-hop IP address is needed.

This helped a lot: https://community.fortinet.com/t5/FortiGate/Technical-Tip-Configure-policy-routes-for-route-based-in...

But I haven't yet been able to solve the VRF problem.

I am just avoiding to work with these virtual routing tables on Forti platzforms...

So the problem of the actual post hasn't been solved yet!