Vdom issue

[Nailed]

Hi all,

I'm having an issue with my routing and policy's on the fortigate 800D. We are migrating from Juniper(screenOs) to the fortigate 800D. In the Juniper firewall we've created multiple virtual routers to make multiple routing instances for other company's connected to our infrastructure. We are not connected to the internet just to a couple third-party’s. Because the fortigate doesn't support virtual routers we are using Vdoms. Well there is our problem i cant find a way to setup my policy’s and routing through multiple vdoms. For example: 

We have hosts in the root-vdom that need to connect to hosts in Vdom-a. There are some policy’s applying to that as well. I have used the forticonverter software to convert the complete rule base of the Juniper firewall to fortigate CLI commands. If i want to import the rules i receive an error because there are rules from an interface within the root-vdom to vdom-a but i cant select an interface that is used in another vdom. 

I have tried to create a vlink but that doesn’t seem to fix my problem either, or i misconfigured that one. 

Do you guys have some idea's how to make a static route between vdom's and setup cross-vdom policy's? 

Thanks in advance 

[emnoc]

You need to look at inter-vdom links or physical links ( the former reduce waste of ports ). Check out a post I did a few years back just about this

http://socpuppet.blogspot.com/2014/09/a-meshed-vdom-transparent-using-inter.html

ken

[Nailed]

Thanks Ken, i found your post very helpful. I still have one issue with the configuration.

In V-domA i have subnet 10.x.x.x/30 

In V-domB i have subnet 213.x.x.x/28 

On the Vlink between both Vdoms i assigned 172.16.1.10 for vdomA and 172.16.1.50 for vdomB.

Afterwards i created in both vdom a static route for the subnets above mentioned and point the gateway to the vlink interface of the other Vdom. So VdomA has a route with gateway 172.16.1.50 and VdomB points to 172.16.1.10.

When i do a traceroute i have no issues with reaching both gateway addresses from both vdoms, if i try to reach 213.x.x.x network he doesn’t even pass the 172.16.1.50 interface... All addresses are listed in my routing table so at this point I have no clue why ping or traceroute doesn’t work (its enabled on all interfaces and even allowed trough a policy.) 

You guys have any clue what could be the issue?

Thanks in advance

[emnoc]

What I would look at?

1:the cli  diag debug flow did you enable and diagnostic trace

2: fwpolicy correct ( again #1 will show you what firewallpolicy your hitting or not ) and the fwpolic would look like this

config firewall policy

    edit 0

       set srcintf  LAN_NETWORK

       set dstintf  VdomA-link

       set srcadr  LAN_NETWORK

       set dstaddr  REMOTE_NETWORK

       set action  accept

       set scehdule always

       set services  BLAH BLAH BLAH

  end

repeat at the other vdom

3: check duplicate routes entries for the remote-subnets

[Nailed]

Thanks for your help.

The debug diag command does'nt get me any further. I have my configured as above. now i made a BGP connection on the Vlink so i cant mess up my static routes. 

There are no double entries in both vdoms. there are only 2 subnets connected at this point because of the migration. 

The point i dont understand is that he now shows the right subnet in the routing monitor of VdomA & VdomB with the right gateway, he is able to ping that gateway but still unable to access that subnet behind it. 

[emnoc]

if your using the debug command correctly it will

1: show you the route

2: matching firewpolicy

3: deny/accept

Do this

diag debug reset

diag debug enable

! x.x.x.x ==  vdomA or vdomB host

diag debug flow  filter addr x.x.x.x

diag debug flow filter proto 1

diag debug flow  show console enable

diag debug flow  trace start 10

than start traffic { ping } to the remote vdom from  vdomA to vdomB,

Do you get anything in the trace? Was SNAT involved? Did it match your expected firewall policy ? Was it a success or failure?

Traffic can't flow thru a firewall and not make a trace.

After you  have answer the above reset and disable the diag

diag debug reset

diag debug disable

Ken

[Nailed]

While doing the debug as you said i found out something interesting. If i try to access the ip address of the host in VdomB from VdomA he tries to find a route through "root". if i ping the vdomb side of the Vlink he finds his route through the Vlink. so i dont know why he isnt sending traffic for the vdomb subnet trough the vlink but that is where my problem exists. Now i need to find a way to send this traffic over the vlink. 

my static route on vdomA is : Dest subnet 172.16.1.0/24 (changed it to make it a bit easier for me while testing this is vdomB) device: Vlink in vdomA and gateway 10.1.1.2(vlink vdomB) 

In VdomB i have the exact oposit.

These are the results: 

id=20085 trace_id=105 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.71.121.66:5376->176.16.1.10:2048) from port13. type=8, code=0, id=5376, seq=0."

id=20085 trace_id=105 func=init_ip_session_common line=4944 msg="allocate a new session-0012e8b3"

id=20085 trace_id=105 func=vf_ip_route_input_common line=2586 msg="find a route: flag=00000000 gw-176.16.1.10 via root"

id=20085 trace_id=106 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.71.121.66:5376->176.16.1.10:2048) from port13. type=8, code=0, id=5376, seq=1."

id=20085 trace_id=106 func=init_ip_session_common line=4944 msg="allocate a new session-0012e8b5"

id=20085 trace_id=106 func=vf_ip_route_input_common line=2586 msg="find a route: flag=00000000 gw-176.16.1.10 via root"

id=20085 trace_id=107 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.71.121.66:5376->176.16.1.10:2048) from port13. type=8, code=0, id=5376, seq=2."

id=20085 trace_id=107 func=init_ip_session_common line=4944 msg="allocate a new session-0012e8b6"

id=20085 trace_id=107 func=vf_ip_route_input_common line=2586 msg="find a route: flag=00000000 gw-176.16.1.10 via root"

id=20085 trace_id=108 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.71.121.66:5376->176.16.1.10:2048) from port13. type=8, code=0, id=5376, seq=3."

id=20085 trace_id=108 func=init_ip_session_common line=4944 msg="allocate a new session-0012e8bd"

id=20085 trace_id=108 func=vf_ip_route_input_common line=2586 msg="find a route: flag=00000000 gw-176.16.1.10 via root"

id=20085 trace_id=109 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.71.121.66:5376->176.16.1.10:2048) from port13. type=8, code=0, id=5376, seq=4."

id=20085 trace_id=109 func=init_ip_session_common line=4944 msg="allocate a new session-0012e8bf"

id=20085 trace_id=109 func=vf_ip_route_input_common line=2586 msg="find a route: flag=00000000 gw-176.16.1.10 via root"

id=20085 trace_id=110 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.71.121.66:5632->10.1.1.2:2048) from port13. type=8, code=0, id=5632, seq=0."

id=20085 trace_id=110 func=init_ip_session_common line=4944 msg="allocate a new session-0012e9a1"

id=20085 trace_id=110 func=vf_ip_route_input_common line=2586 msg="find a route: flag=04000000 gw-10.1.1.2 via Test R-G0"

id=20085 trace_id=110 func=fw_forward_handler line=697 msg="Allowed by Policy-3:"

id=20085 trace_id=111 func=print_pkt_detail line=4793 msg="vd-GSI received a packet(proto=1, 10.71.121.66:5632->10.1.1.2:2048) from Test R-G1. type=8, code=0, id=5632, seq=0."

id=20085 trace_id=111 func=init_ip_session_common line=4944 msg="allocate a new session-0012e9a2"

id=20085 trace_id=111 func=vf_ip_route_input_common line=2586 msg="find a route: flag=80000000 gw-10.1.1.2 via GSI"

id=20085 trace_id=112 func=print_pkt_detail line=4793 msg="vd-GSI received a packet(proto=1, 10.1.1.2:5632->10.71.121.66:0) from local. type=0, code=0, id=5632, seq=0."

id=20085 trace_id=112 func=resolve_ip_tuple_fast line=4857 msg="Find an existing session, id-0012e9a2, reply direction"

id=20085 trace_id=113 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.1.1.2:5632->10.71.121.66:0) from Test R-G0. type=0, code=0, id=5632, seq=0."

id=20085 trace_id=113 func=resolve_ip_tuple_fast line=4857 msg="Find an existing session, id-0012e9a1, reply direction"

id=20085 trace_id=113 func=vf_ip_route_input_common line=2586 msg="find a route: flag=04000000 gw-10.71.121.66 via port13"

id=20085 trace_id=113 func=npu_handle_session44 line=1028 msg="Trying to offloading session from Test R-G0 to port13, skb.npu_flag=00000000 ses.state=00800204 ses.npu_state=0x00040001"

id=20085 trace_id=114 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.71.121.66:5632->10.1.1.2:2048) from port13. type=8, code=0, id=5632, seq=1."

id=20085 trace_id=114 func=resolve_ip_tuple_fast line=4857 msg="Find an existing session, id-0012e9a1, original direction"

id=20085 trace_id=114 func=npu_handle_session44 line=1028 msg="Trying to offloading session from port13 to Test R-G0, skb.npu_flag=00000400 ses.state=00800204 ses.npu_state=0x00040001"

[emnoc]

Did you do the "src pings from a host within the VDOM and not root" ?

Without seeing your topology an cfg, it hard to gather but double check the intervdomlinks two endppoints you probably could do it numerous ways.

eg see visio

[Nailed]

I tried both ways. The idea of our topology is that our own hosts are located in root. Third party hosts get their own vdom. See also drawing with the current configuration. 

With the output of the debug i think i'm a lot closer to the solution as i was 2 days ago. 

[Nailed]

Thanks very much for your help! 

while i was running a couple test with the diag debug i found out that traffic was denied by policy 0 (implicit deny).

This complete idiot totaly forgot to allow acces from outside to inside, i only configured a rule from inside to outside on both ends *heavy facepalm*. 

My problem is solved now and i can move on with migration, thanks for your advice in this case!