routing native IPv6 through a fortigate

[vaxxus]

I have some trouble getting ipv6 running behind my fortigate in native mode (meaning without NAT).

It works fine with NAT66, but the moment I turn NAT of on the firewall policy everything is dead.

I have a /48 from my provider and configured a /64 for the internal lan where I took on address for my test computer.

The only v6 route I setup is a default ::/0 to the router of my provider (which as stated seems to be all I have to do to get NAT66 running).

I can ping6 my external fortigate address, but not my internal computer, even though I trid a basic all/all/all policy for that as well. As the line in the policy doesnt show any traffic at all I suspect some routing issues and something I still have to setup, but I have no clue what is missing, as the monitoring section in the fortigate states a number of v6 routes saying "connected" (one of them being the internal v6 /64 going to the lan interface.

any pointers appreciated.

[emnoc]

The diag debug flow filter6 is your  cli cmd to trouble-shoot  your fw-policies. I use it as my 1st choice and in all  t-shooting.

Back on routing, grab the route ipv6 table . Validate the ipv6 route tables. Can you ping the SRC/DST hosts from  the firewall? Does the routing look good ? Are  the DNS options correct ? Are the host dual-stack ? If yes, are the preference for ipv4 or ipv6 ? Also are the hosts using the correct gateway? How do they get the gw ( static, slaac, dhcpv6srv)

FWIW, ipv6 is no harder than  ipv4. Actually based on my experience of using ipv6 on firewall and primary  Fortigate, it's actually much better imho.

http://socpuppet.blogspot.com/2013/03/flow-diagnostic-fortigate.html

Ken

[vaxxus]

thanks for the pointer, but I am afraid it doesnt get me anywhere, because I still dont see whats wrong. But to answer your questions first:

1) I can ping the outside host fine from the firewall (when it uses its external interface)

2) I can NOT ping the outside host from the firewall when I use the -I internalinterface parameter, then I get the error "connect: Network is unreachable"

3) routing "looks" good in the way that I can see routes for my internal lan and the outgoing default to the router of the provider

get router info6 routing-table IPv6 Routing Table Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 I - IS-IS, B - BGP * - candidate default

Timers: Uptime

S* ::/0 [10/0] via 2003:54:19::1, port2, 21:41:19 C ::1/128 via ::, root, 3d01h01m C 2003:54:19::/64 via ::, port2, 3d01h01m C 2003:54:19:1::/64 via ::, port1, 3d01h01m C 2003:54:19:2::/64 via ::, dmz, 3d01h01m C fe80::/10 via ::, port2, 3d01h01m C fe80::/64 via ::, dmz, 3d01h01m K ff00::/8 via ::, wlanguest, 3d01h01m

4) dns is not relevant atm as I am still in the stage of trying raw ip addresses :)

5) test host is dual stacked and ip6 is set via manual option to  

IPv6 Address. . . . . . . . . . . : 2003:54:19:1::99(Preferred)

Default Gateway . . . . . . . . . : 2003:54:19:1::8

(which is the fortigate in question)

hier is the trace from my test; above the --- is the try without NAT enable (not working) and below the --- is NAT enabled in the rule and the ping is working.

2017-07-14 13:30:43 id=20085 trace_id=1005015 func=resolve_ip6_tuple_fast line=3438 msg="vd-root received a packet(proto=58, 2003:54:19:1::99:1->2001:4860:4860::8888:128) from port1." 2017-07-14 13:30:43 id=20085 trace_id=1005015 func=resolve_ip6_tuple line=3537 msg="allocate a new session-000599ab" 2017-07-14 13:30:43 id=20085 trace_id=1005015 func=vf_ip6_route_input line=921 msg="find a route: gw-2003:54:19::1 via port2 err 0 flags 00000003" 2017-07-14 13:30:43 id=20085 trace_id=1005015 func=fw6_forward_handler line=322 msg="Check policy between port1 -> port2" 2017-07-14 13:30:43 id=20085 trace_id=1005015 func=fw6_forward_handler line=448 msg="Allowed by Policy-4:" 2017-07-14 13:30:48 id=20085 trace_id=1005016 func=resolve_ip6_tuple_fast line=3438 msg="vd-root received a packet(proto=58, 2003:54:19:1::99:1->2001:4860:4860::8888:128) from port1." 2017-07-14 13:30:48 id=20085 trace_id=1005016 func=resolve_ip6_tuple_fast line=3463 msg="Find an existing session, id-000599ab, original direction" 2017-07-14 13:30:48 id=20085 trace_id=1005016 func=ip6_session_install_npu_session line=274 msg="npu session intallation succeeded" ---- 2017-07-14 13:31:58 id=20085 trace_id=1005017 func=resolve_ip6_tuple_fast line=3438 msg="vd-root received a packet(proto=58, 2003:54:19:1::99:1->2001:4860:4860::8888:128) from port1." 2017-07-14 13:31:58 id=20085 trace_id=1005017 func=resolve_ip6_tuple_fast line=3463 msg="Find an existing session, id-000599ab, original direction" 2017-07-14 13:31:58 id=20085 trace_id=1005017 func=get_new_addr6 line=768 msg="find NAT: IP-2003:54:19::8, port-8424" 2017-07-14 13:31:58 id=20085 trace_id=1005017 func=fw6_strict_dirty_session_check line=123 msg="SNAT mismatch, drop" 2017-07-14 13:32:03 id=20085 trace_id=1005018 func=resolve_ip6_tuple_fast line=3438 msg="vd-root received a packet(proto=58, 2003:54:19:1::99:1->2001:4860:4860::8888:128) from port1." 2017-07-14 13:32:03 id=20085 trace_id=1005018 func=resolve_ip6_tuple line=3537 msg="allocate a new session-00059aea" 2017-07-14 13:32:03 id=20085 trace_id=1005018 func=vf_ip6_route_input line=921 msg="find a route: gw-2003:54:19::1 via port2 err 0 flags 00000003" 2017-07-14 13:32:03 id=20085 trace_id=1005018 func=fw6_forward_handler line=322 msg="Check policy between port1 -> port2" 2017-07-14 13:32:03 id=20085 trace_id=1005018 func=get_new_addr6 line=768 msg="find NAT: IP-2003:54:19::8, port-8424" 2017-07-14 13:32:03 id=20085 trace_id=1005018 func=fw6_forward_handler line=448 msg="Allowed by Policy-4: SNAT" 2017-07-14 13:32:04 id=20085 trace_id=1005019 func=resolve_ip6_tuple_fast line=3438 msg="vd-root received a packet(proto=58, 2003:54:19:1::99:1->2001:4860:4860::8888:128) from port1." 2017-07-14 13:32:04 id=20085 trace_id=1005019 func=resolve_ip6_tuple_fast line=3463 msg="Find an existing session, id-00059aea, original direction" 2017-07-14 13:32:04 id=20085 trace_id=1005019 func=ip6_session_install_npu_session line=274 msg="npu session intallation succeeded"

port1 is the lan port, port2 is the wan port.

[vaxxus]

another thing I find odd. Trying to get a response from outside (pinging into my net from outside). Shouldnt that at least give me some trace response even though I do not have a valid policy rule set? Like some sort of drop trace saying "yes there was a packet but it got dropped because not allowed"?

Instead there is silence. The only trace lines I can force to appear is when I ping the firewall directly (2003:54:19::8), but trying something like 2003:54:19:1::1 (internal interface) does not give me any trace line at all.

Could it be, that I have to somehow announce to the provider router that there are subnets behind my 2003:54:19::1, so that the packets even reach my firewall? Now that I type that I faintly remember that doing on the old sonicwall somehow maybe.

With the v4 addresses we have I know that all get thrown at my firewall and I have to sort them out and so far I was under the impression that is true for the whole 2003:54:19:: as well, but then everything I type behind that should net a traceline in my debugs, no?

[emnoc]

Will if the internet can't see you,  than that would be a big clue  & a bigger issue  ;)

What I would do;

1> go to a NTT looking-glass and select any ipv6 router in any region

2> conduct a ping to A> your wan ipv6 address ( ensure set allowaccess ping is enable for that interface )

# examples

#

config sys interface

   config ipv6

   edit port1

        set ip6-allowaccess  ping

   end

3> does it pass? if yes , than goto #4

4> enable one of your  /64  subnets that's giving you  trouble on a loopback interface, enable set allow access ping , and repeat #2 but this time to the ipv6-address on the loopback ( ensure you have a policy6 for  traffic service ICMP6 to  the loopback )

# firewall6 policy  with traffic from intenet to your loopback  named loop6

#

#

config firewall policy6     edit 1           set action accept

          set service all

          set  schedule alway

          set srcintf  port1

          set dstint  loop6

          set srcaddr all

          set dstaddr all

          set comment "allow for pings from  NTT looking glass"

 end

If all of these are good than it's not routing from a internet stand-point if not STOP and fix  your routing between SP and your firewall ( bgp advertisements, static6 routes,etc.....)

Again routing ipv6 is no different than ipv4 the same  functions  ( next-hop, NRLI, etc......)

Ken Felix

[vaxxus]

well, it might be "no different" from your perspective for someone who is doing it all days, but let me tell you that it is totally different from my pov :)

I can configure v4 nets just fine, but I seem to fail on an epic scale when it comes to v6, because its simply not working.

So I configured that loop interface and managed to ping the address from inside my lan (and even from my dmz), but I was unable ping it from outside.

So while traffic into the fortinet seems to work just fine routing wise, it stops working the moment it should cross the fortigate, even though I have setup the policy to allow everything. (Tried pinging the lan machine from the dmz machine and vice versa and this fails as well).

When I check the routing monitor I see v6 lines stated for every interface, so I assumed that I do not have to put up extra static routing rules for these. Maybe this is wrong and I have to setup a route for each network?

Just tried, did'nt help at all :(

I really dont get where I fail, it seemed a lot easier on the sonicwall.

I can ping my external address from outside just fine, but I can not ping the loopback address, even though I have set the policy right. That would mean that routing sucks at some point, right?

Also don't get why a ping works to the outside fortigate address but a traceroute6/path6 fails miserably.

Shouldnt the fortigate behave the same?

ping6 2003:54:19::8 PING 2003:54:19::8(2003:54:19::8) 56 data bytes 64 bytes from 2003:54:19::8: icmp_seq=1 ttl=56 time=7.36 ms

traceroute6 2003:54:19::8 traceroute to 2003:54:19::8 (2003:54:19::8) from 2a01:4f8:171:1445::2, 30 hops max, 24 byte packets 1 2a01:4f8::a:17:a (2a01:4f8::a:17:a) 0.324 ms 0.351 ms 0.299 ms 2 core24.fsn1.hetzner.com (2a01:4f8:0:3::169) 0.363 ms 0.749 ms 1.194 ms 3 core1.fra.hetzner.com (2a01:4f8:0:3::1bd) 5 ms 5.403 ms 4.978 ms 4 et-0-0-47.cr10-fra2.ip6.gtt.net (2001:668:0:3::2000:1421) 6.547 ms 5.029 ms 5.189 ms 5 2001:668:0:2:ffff:0:5995:b809 (2001:668:0:2:ffff:0:5995:b809) 5.133 ms 5.162 ms 5.123 ms 6 2003:0:1308:800e::1 (2003:0:1308:800e::1) 6.53 ms 7.256 ms 7.929 ms 7 2003:0:8305:8000::1 (2003:0:8305:8000::1) 8.153 ms 7.76 ms 8.447 ms 8 2003:0:8305:8003::1 (2003:0:8305:8003::1) 17.085 ms 9.531 ms 9.886 ms

9 * * * 10 * * *

[... and so on]

which is strange, because if I traceroute to the isp router just in front of the fortigat I get:

traceroute6 2003:54:19::1 traceroute to 2003:54:19::1 (2003:54:19::1) from 2a01:4f8:171:1445::2, 30 hops max, 24 byte packets 1 2a01:4f8::a:17:a (2a01:4f8::a:17:a) 0.329 ms 0.305 ms 0.263 ms 2 core24.fsn1.hetzner.com (2a01:4f8:0:3::169) 0.331 ms 0.304 ms 0.363 ms 3 core1.fra.hetzner.com (2a01:4f8:0:3::1bd) 4.998 ms 5.02 ms 4.971 ms 4 et-0-0-47.cr10-fra2.ip6.gtt.net (2001:668:0:3::2000:1421) 5.051 ms 5.459 ms 5.168 ms 5 2001:668:0:2:ffff:0:5995:b809 (2001:668:0:2:ffff:0:5995:b809) 5.103 ms 5.075 ms 5.083 ms 6 2003:0:1308:800e::1 (2003:0:1308:800e::1) 6.298 ms 7.415 ms 7.99 ms 7 2003:0:8305:8000::1 (2003:0:8305:8000::1) 6.235 ms 6.214 ms 6.181 ms 8 2003:54:19::1 (2003:54:19::1) 18.695 ms 10.187 ms 9.545 ms

isnt that already wrong?

[vaxxus]

ok, so I experimented a bit more and in that, I changed the external ip address of my fortigate to something else with the result now that nothing to the outside world is working anymore, even after putting it back to the address which was working (as you can see in my last post, pinging to 2003:54:19::8 was possible and now its not.

So it seems to me, that the isp router somehow has a problem to understand what is behind him and what not. I know that there are these magic routing things like rip, bgp and ospf which seems todo some kind of auto routing announcement stuff to neighbours like there is a dhcp server giving out address to anyone who asks, but frankly I have no clue about that, as I only had todo basic manual routing stuff in v4, so never took a closer look at these things. I already figured out that the gui isnt a fan of putting anything v6 related into any of these sections, so I guess I have todo it on the cli, IF thats actually something I need todo in order to get that telekom router to route my stuff again...

[emnoc]

On traceroute6 failing it might be UDP and not ICMP.

On the others issues, did you peel off one of the networks that you have routed to you and temp place it on a loop-interface & open a any....Any policy6 for traffic to it? Than try that same ping6 and traceroute6 to that loopback address. Does it pass?

Do you have a filter6  diag debug flow enabled? if yes does it see any matches?

use the following for help with ipv6 routing.

https://us.ntt.net/support/looking-glass/

Tracing the route to 2003:54:19::8

 1  ae-5.r20.atlnga05.us.bb.gin.ntt.net (2001:418:0:2000::10e) 1 msec 0 msec 0 msec
 2  ae-4.r22.asbnva02.us.bb.gin.ntt.net (2001:418:0:2000::1b9) 13 msec 12 msec 12 msec
 3  ae-1.r05.asbnva02.us.bb.gin.ntt.net (2001:418:0:2000::19) 14 msec 13 msec 13 msec
 4  ae-0.a01.asbnva02.us.bb.gin.ntt.net (2001:418:0:2000::2cd) 26 msec 12 msec 13 msec
 5  2003:3c0:1600:4::1 12 msec 12 msec 15 msec
 6  2003:0:8305:8000::1 102 msec 102 msec 101 msec
 7  2003:0:8305:8003::1 106 msec 112 msec 105 msec
 8   *  *  * 
 9   *  *  * 
 10  *  *  * 
 11  *  *  * 

[vaxxus]

trying to ping something outside now gives me this:

2017-07-18 08:14:03 id=20085 trace_id=1029454 func=resolve_ip6_tuple line=3537 msg="allocate a new session-000ed1fd" 2017-07-18 08:14:03 id=20085 trace_id=1029454 func=vf_ip6_route_input line=921 msg="find a route: gw-:: via root err -101 flags 00200200" 2017-07-18 08:14:04 id=20085 trace_id=1029455 func=resolve_ip6_tuple_fast line=3438 msg="vd-root received a packet(proto=58, 2003:54:19:1::99:1->2a01:4f8:171:1445::2:128) from port1." 2017-07-18 08:14:04 id=20085 trace_id=1029455 func=resolve_ip6_tuple line=3537 msg="allocate a new session-000ed207"

so its seems that I suddenly do not even have a default gateway anymore? But if I look up the routing table it says:

get router info6 routing-table IPv6 Routing Table Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 I - IS-IS, B - BGP * - candidate default

Timers: Uptime

S* ::/0 [10/0] via 2003:54:19::1, port2, 15:51:59 C ::1/128 via ::, root, 6d19h32m C 2003:54:19::/64 via ::, port2, 16:24:26 C 2003:54:19:1::/64 via ::, port1, 16:24:50 C 2003:54:19:2::/64 via ::, dmz, 6d19h32m C 2003:54:19:3::/64 via ::, loopback, 16:37:09 C fe80::/10 via ::, port2, 6d19h32m C fe80::/64 via ::, dmz, 6d19h32m K ff00::/8 via ::, wlanguest, 6d19h32m

so this should go to port2 to 2003:54:19::1 (which is the isp router).

Also looking at that routing table output closely, shouldnt the "via ::" acutally read "via ipv6addressofofrtigateinterface" ?

at the client, the ping says "Destination unreachable" which normaly means "no routing" right? But I do not see where that routing is actually missing when the table says "default go to 2003:54:19::1" and that one is reachable directly from the fortigate shell (but not from the client)?

Also I get the debug output when trying to reach that outer system above, but not when I try the isp routerm, thats odd as well, as the diag debug flow filter6 says "any" to everything, therefore I would expect to get anything.

This is driving me mad. I hate when machine become human and act not logically, that should be reserverd for us dummies...

[vaxxus]

and to answer your questions.

lan -> loopback OK

dmz -> loopback OK

lan -> dmz OK

dmz -> lan OK

trying to reach something outside my networks I get this error:

2017-07-18 11:07:23 id=20085 trace_id=1030015 func=resolve_ip6_tuple_fast line=3438 msg="vd-root received a packet(proto=58, 2003:54:19:2::99:1->2a01:4f8:171:1445::2:128) from dmz." 2017-07-18 11:07:23 id=20085 trace_id=1030015 func=resolve_ip6_tuple line=3537 msg="allocate a new session-000f80c0" 2017-07-18 11:07:23 id=20085 trace_id=1030015 func=vf_ip6_route_input line=921 msg="find a route: gw-:: via root err -101 flags 00200200" 2017-07-18 11:07:24 id=20085 trace_id=1030016 func=resolve_ip6_tuple_fast line=3438 msg="vd-root received a packet(proto=58, 2003:54:19:2::99:1->2a01:4f8:171:1445::2:128) from dmz."

pinging the external interface of the fortigate from the dmz machine does work though, but I can not ping the isp provider from my dmz test machine. The strange thing there is, that the debug log doesnt even show an attempt.