Descritpion
GRE encapsulation is used to tunnel intercepted traffic between the 2 FortiGates.
This article provides configuration samples with debug command output samples.
Scope
WCCP client feature has been introduced in 4.0 MR2 release.
FortiGate versions 4.3 to 5.0.
Diagram
The following diagram illustrates the example provided in this article.
Expectations, Requirements
Expectations:
- ion-mvm-14 requests HTTP traffic on the Internet (tcp port 80), browser is not configured to use a proxy.
- FG500A-2 acting as a WCCP server intercepts the HTTP traffic with a WCCP policy and redirects traffic to WCCP client (FG310B-4) through a GRE tunnel.
- FG310B-4 gets the content from the Internet (using a policy on FG500A-2) and returns the traffic to FG500A-2 via the GRE tunnel.
- FG500A-2 re-injects the traffic towards the ion-mvm-14
WCCP highlights:
WCCP stands for "Web Cache Communication Protocol". The goal is to transparently redirect user traffic to a proxy (that is: NO explicit web proxy configured on client web browser).
- Client traffic going through the WCCP server is intercepted and redirected to a web proxy.
- The web proxy gets the content if required and services the client by sending the content back to the WCCP Server.
- The WCCP server is in charge of retransmitting the content from the web proxy to the client.
- WCCP is defined as an RFC draft
- It has built-in load-balancing and fault tolerance
- WCCP protocol uses UDP port 2048 to communicate between WCCP server and client
- WCCP has 2 types of services (well-known service id=0 for http and dynamic services (from id 51 to 255)
- WCCP allows 2 types of tunneling mechanisms ( GRE and L2 Forwarding)
Configuration
FG500A-2 CLI configuration (only the relevant configuration is shown)
config system interface edit "port2" set vdom "root" set ip 192.168.182.158 255.255.254.0 set allowaccess ping https ssh http telnet set type physical next edit "lan" set vdom "root" set ip 10.100.0.158 255.255.254.0 set allowaccess ping https ssh http telnet set type physical set wccp enable (#1) next end config firewall address edit "WCCP_Client" set subnet 10.100.1.32 255.255.255.255 next end config firewall policy edit 2 set srcintf "lan" set dstintf "port2" set srcaddr "WCCP_Client" set dstaddr "all" set action accept set schedule "always" set service "ANY" set nat enable next edit 1 set srcintf "lan" set dstintf "port2" set srcaddr "all" set dstaddr "all" set action accept set wccp enable (#2) set schedule "always" set service "ANY" set nat enable next end config router static edit 1 set device "port2" set gateway 192.168.183.254 next end config system wccp (#3) edit "100" set router-id 10.100.0.158 set server-list 10.100.0.0 255.255.254.0 next end |
Keys of the configuration:
- #1 : the interface toward the proxy must be 'set wccp enable' to designate on which interface the GRE tunnel should be starting
- #2 : the policy intercepting the traffic must be marked 'set wccp enable'
- #3 : WCCP dynamic server 100 is configured,
server advertised its ip address 10.10.0.158 as wccp router-id,
authorized WCCP client id should be part of network 10.100.0.0/23
FG310B-4 CLI configuration (only the relevant configuration is shown)
config system interface edit "port5" set vdom "root" set ip 10.100.1.32 255.255.254.0 set allowaccess ping https ssh http telnet set type physical set wccp enable (#1) next end config system settings set wccp-cache-engine enable (#2) end config firewall policy edit 1 set srcintf "w.root" (#3) set dstintf "port5" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ANY" set nat enable next end config router static edit 1 set device "port5" set gateway 10.100.0.158 next end config system wccp (#4) edit "100" set cache-id 10.100.1.32 set router-list "10.100.0.158" set ports 80 443 set protocol 6 next end |
Keys of the configuration:
- #1 : the interface toward the wccp server must be 'set wccp enable' to designate on which interface the GRE tunnel should be starting
- #2 : WCCP is configured as a client (ie: cache)
- #3 : the source interface from which the GRE tunnel is bound to is format has "w.<vdom_name>" so w.root in this example this interface is created automatically when WCCP client feature is enabled (step #2)
- #4 : WCCP dynamic server 100 is configured (matching the WCCP server config),
WCCP client advertises its ip address 10.100.1.32 as WCCP client-id (should be within wccp server server-list definition)
WCCP server ip configured is 10.100.0.158
WCCP dynamic service 100 is configured to intercept tcp (proto 6) port 80 and 443
Verification
WCCP debug
# diagnose test application wccpd <Integer>
Where <Integer> is for :
1. Dump wccp stats
2. Dump wccp config
3. Dump wccp2 cache servers
4. Dump wccp2 services
5. Dump wccp2 assignment
6. Dump wccp2_cache status
FG500A-2 :
FGT500A-2 # diagnose test application wccpd 1 vdoms=1 pkts=0 FGT500A-2 # diagnose test application wccpd 2 vdom-root: work mode:router working NAT first_phy_id=6 interface list: intf=lan, gid=6 phy_id=6 service list: service: 100, router_id=10.100.0.158, group=0.0.0.0, auth(no) access:10.100.0.0/255.255.254.0 forward=1 return=1, assign=1. erouter_id=10.100.0.158 FGT500A-2 # diagnose test application wccpd 3 service-100 in vdom-root: num=1, usable=1 cache server ID: len=44, addr=10.100.1.32, weight=0, status=0 rcv_id=232, usable=1, fm=1, nq=0, dev=6(k6), to=10.100.0.158 ch_no=0, num_router=1: 10.100.0.158 FGT500A-2 # diagnose test application wccpd 4 service-100 in vdom-root: total_servers=1, usable_servers=1, assign_m=1, rtun_m=1, wcid_len=48, rcv_id=232, ch_no=1 ID=100, type=1, pri=0, pro=6 f=00000012 Port: 80 443 num-routers=1: 10.100.0.158 FGT500A-2 # diagnose test application wccpd 5 service-100 in vdom-root: installed key: ip=10.100.1.32, change-number=1 cache_list: 1 0. 10.100.1.32 primary assignment: key=10.100.1.32 change-number=1 num_routers=1 router element[0]: router_id=10.100.0.158, receive_id=4, ch_no=1 cache-server-num=1, format=not standard: 10.100.1.32 buckets: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |
FG310B-4 :
FGT310B-4 # diagnose test application wccpd 1 vdoms=1 pkts=0 FGT310B-4 # diagnose test application wccpd 2 vdom-root: work mode:cache working NAT first_phy_id=6 interface list: intf=port5, gid=6 phy_id=6 service list: service: 100, cache_id=10.100.1.32, group=0.0.0.0, auth(no) forward=1, return=1, assign=1. router list: 10.100.0.158 port list: 80 443 ecache_id=10.100.1.32 FGT310B-4 # diagnose test application wccpd 3 FGT310B-4 # diagnose test application wccpd 4 FGT310B-4 # diagnose test application wccpd 5 FGT310B-4 # diagnose test application wccpd 6 service-100 in vdom-root erouter_list: 1 routers in total 0. 10.100.0.158 receive_id:16950 change_number:1 cache servers seen by this router: 0. 10.100.1.32 weight:0 (*Designated Web Cache) |
Firewall session samples:
FG500A-2:
session info: proto=17 proto_state=01 duration=166269 expire=176 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4 origin-shaper= reply-shaper= per_ip_shaper= ha_id=0 hakey=446 policy_dir=0 tunnel=/ state=local may_dirty rem statistic(bytes/packets/allow_err): org=2394076/16624/1 reply=8244572/16624/1 tuples=2 orgin->sink: org pre->in, reply out->post dev=6->10/10->6 gwy=10.100.0.158/0.0.0.0 hook=pre dir=org act=noop 10.100.1.32:2048->10.100.0.158:2048(0.0.0.0:0) hook=post dir=reply act=noop 10.100.0.158:2048->10.100.1.32:2048(0.0.0.0:0) misc=0 policy_id=0 id_policy_id=0 auth_info=0 chk_client_info=0 vd=0 serial=00000089 tos=ff/ff app_list=0 app=0 dd_type=0 dd_rule_id=0 per_ip_bandwidth meter: addr=10.100.1.32, bps=2257041 ==> WCCP session between server and client session info: proto=6 proto_state=01 duration=21 expire=3600 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4 origin-shaper= reply-shaper= per_ip_shaper= ha_id=0 hakey=28063 policy_dir=0 tunnel=/ state=wccp may_dirty nlb rem statistic(bytes/packets/allow_err): org=213244/3953/1 reply=18836335/12566/1 tuples=2 orgin->sink: org pre->post, reply pre->post dev=18->6/6->18 gwy=10.100.0.158/192.168.182.158 hook=post dir=org act=snat 192.168.182.158:60085->192.168.183.1:80(10.100.1.32:56341) hook=pre dir=reply act=dnat 192.168.183.1:80->10.100.1.32:56341(192.168.182.158:60085) pos/(before,after) 0/(0,0), 0/(0,0) misc=0 policy_id=1 id_policy_id=0 auth_info=0 chk_client_info=0 vd=0 serial=000092b5 tos=ff/ff app_list=0 app=0 dd_type=0 dd_rule_id=0 wccp: rmt=10.100.0.158, local=10.100.1.32, dev=6, pri=222, alt=0 GRE-F GRE-R dyn per_ip_bandwidth meter: addr=192.168.182.158, bps=1721768 ==> client traffic intercepted and redirected to WCCP GRE tunnel with interface indexes : if=port2 family=00 type=1 index=3 mtu=1500 link=0 master=0 ref=21 state=start present flags=up broadcast run promsic multicast if=lan family=00 type=1 index=6 mtu=1500 link=0 master=0 ref=14 state=start present flags=up broadcast run promsic multicast if=root family=00 type=772 index=10 mtu=16436 link=0 master=0 ref=16 state=start present flags=up loopback run |
FG310B-4:
session info: proto=17 proto_state=01 duration=166232 expire=177 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3 origin-shaper= reply-shaper= per_ip_shaper= ha_id=0 hakey=446 policy_dir=0 tunnel=/ state=local statistic(bytes/packets/allow_err): org=2394212/16625/1 reply=8244572/16624/1 tuples=2 orgin->sink: org out->post, reply pre->in dev=13->6/6->13 gwy=0.0.0.0/10.100.1.32 hook=out dir=org act=noop 10.100.1.32:2048->10.100.0.158:2048(0.0.0.0:0) hook=in dir=reply act=noop 10.100.0.158:2048->10.100.1.32:2048(0.0.0.0:0) misc=0 policy_id=0 id_policy_id=0 auth_info=0 chk_client_info=0 vd=0 serial=000007db tos=ff/ff app_list=0 app=0 dd_type=0 dd_rule_id=0 per_ip_bandwidth meter: addr=10.100.1.32, bps=6000 ==> WCCP session between Client and Server session info: proto=6 proto_state=01 duration=21 expire=3600 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4 origin-shaper= reply-shaper= per_ip_shaper= ha_id=0 hakey=28063 policy_dir=0 tunnel=/ state=wccp may_dirty nlb rem statistic(bytes/packets/allow_err): org=213244/3953/1 reply=18836335/12566/1 tuples=2 orgin->sink: org pre->post, reply pre->post dev=18->6/6->18 gwy=10.100.0.158/192.168.182.158 hook=post dir=org act=snat 192.168.182.158:60085->192.168.183.1:80(10.100.1.32:56341) hook=pre dir=reply act=dnat 192.168.183.1:80->10.100.1.32:56341(192.168.182.158:60085) pos/(before,after) 0/(0,0), 0/(0,0) misc=0 policy_id=1 id_policy_id=0 auth_info=0 chk_client_info=0 vd=0 serial=000092b5 tos=ff/ff app_list=0 app=0 dd_type=0 dd_rule_id=0 wccp: rmt=10.100.0.158, local=10.100.1.32, dev=6, pri=222, alt=0 GRE-F GRE-R dyn per_ip_bandwidth meter: addr=192.168.182.158, bps=1721768 ==> Traffic received from WCCP server on GRE tunnel (w.root interface) and transmitted to the internet with interface indexes: if=port5 family=00 type=1 index=6 mtu=1500 link=0 master=0 ref=21 state=start present flags=up broadcast run promsic multicast if=root family=00 type=772 index=13 mtu=16436 link=0 master=0 ref=15 state=start present flags=up loopback run if=w.root family=00 type=768 index=18 mtu=16436 link=0 master=0 ref=7 state=start present flags=up p2p run noarp multicast |
Troubleshooting
Commented "debug flow" samples:
FG500A-2:
[1] Client start TCP connection to web server (SYN) 192.168.183.1, packet is natted, WCCP intercepted and pushed to GRE tunnel id=36870 trace_id=5631 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=6, 10.100.1.194:59868->192.168.183.1:80) from lan." id=36870 trace_id=5631 func=resolve_ip_tuple line=3494 msg="allocate a new session-00019c0a" id=36870 trace_id=5631 func=vf_ip4_route_input line=1595 msg="find a route: gw-192.168.183.1 via port2" id=36870 trace_id=5631 func=get_new_addr line=1747 msg="find SNAT: IP-192.168.182.158, port-55672" id=36870 trace_id=5631 func=fw_forward_handler line=472 msg="Allowed by Policy-1: SNAT" id=36870 trace_id=5631 func=__ip_session_run_tuple line=1816 msg="SNAT 10.100.1.194->192.168.182.158:55672" id=36870 trace_id=5631 func=wccp_output line=233 msg="gre_forward" id=36870 trace_id=5631 func=wccp_output line=275 msg="send packet via dev-lan" [3]: WCCP request is seen and natted (policy 2) towards server id=36870 trace_id=5632 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=6, 10.100.1.32:61400->192.168.183.1:80) from lan." id=36870 trace_id=5632 func=resolve_ip_tuple line=3494 msg="allocate a new session-00019c0b" id=36870 trace_id=5632 func=vf_ip4_route_input line=1595 msg="find a route: gw-192.168.183.1 via port2" id=36870 trace_id=5632 func=get_new_addr line=1747 msg="find SNAT: IP-192.168.182.158, port-40828" id=36870 trace_id=5632 func=fw_forward_handler line=472 msg="Allowed by Policy-2: SNAT" id=36870 trace_id=5632 func=__ip_session_run_tuple line=1816 msg="SNAT 10.100.1.32->192.168.182.158:40828" [4]: Server responds (SYN/ACK), packet is returned dnated to WCCP client id=36870 trace_id=5633 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=6, 192.168.183.1:80->192.168.182.158:40828) from port2." id=36870 trace_id=5633 func=resolve_ip_tuple_fast line=3405 msg="Find an existing session, id-00019c0b, reply direction" id=36870 trace_id=5633 func=__ip_session_run_tuple line=1830 msg="DNAT 192.168.182.158:40828->10.100.1.32:61400" id=36870 trace_id=5633 func=vf_ip4_route_input line=1595 msg="find a route: gw-10.100.1.32 via lan" [6]: Response packet received from GRE tunnel and reinjected towards client after a DNAT id=36870 trace_id=5634 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=47, 10.100.1.32:0->10.100.0.158:0) from lan." id=36870 trace_id=5634 func=wccp_gre_decap line=434 msg="feed to dev=port2 wccp=00000000" id=36870 trace_id=5634 func=__ip_session_run_tuple line=1830 msg="DNAT 192.168.182.158:55672->10.100.1.194:59868" id=36870 trace_id=5634 func=vf_ip4_route_input line=1595 msg="find a route: gw-10.100.1.194 via lan" [7] Client Responds with the ACK packet towards the web server and gets intercepted and pushed towards the GRE tunnel id=36870 trace_id=5635 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=6, 10.100.1.194:59868->192.168.183.1:80) from lan." id=36870 trace_id=5635 func=resolve_ip_tuple_fast line=3405 msg="Find an existing session, id-00019c0a, original direction" id=36870 trace_id=5635 func=__ip_session_run_tuple line=1816 msg="SNAT 10.100.1.194->192.168.182.158:55672" id=36870 trace_id=5635 func=wccp_output line=233 msg="gre_forward" id=36870 trace_id=5635 func=wccp_output line=275 msg="send packet via dev-lan" [9]: ACK packet forwarded to Internet id=36870 trace_id=5636 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=6, 10.100.1.32:61400->192.168.183.1:80) from lan." id=36870 trace_id=5636 func=resolve_ip_tuple_fast line=3405 msg="Find an existing session, id-00019c0b, original direction" id=36870 trace_id=5636 func=ipv4_fast_cb line=56 msg="enter fast path" id=36870 trace_id=5636 func=ip_session_run_all_tuple line=4350 msg="SNAT 10.100.1.32->192.168.182.158:40828" .../... same flow continues id=36870 trace_id=5637 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=6, 10.100.1.194:59868->192.168.183.1:80) from lan." id=36870 trace_id=5637 func=resolve_ip_tuple_fast line=3405 msg="Find an existing session, id-00019c0a, original direction" id=36870 trace_id=5637 func=__ip_session_run_tuple line=1816 msg="SNAT 10.100.1.194->192.168.182.158:55672" id=36870 trace_id=5637 func=wccp_output line=233 msg="gre_forward" id=36870 trace_id=5637 func=wccp_output line=275 msg="send packet via dev-lan" id=36870 trace_id=5638 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=6, 10.100.1.32:61400->192.168.183.1:80) from lan." id=36870 trace_id=5638 func=resolve_ip_tuple_fast line=3405 msg="Find an existing session, id-00019c0b, original direction" id=36870 trace_id=5638 func=ipv4_fast_cb line=56 msg="enter fast path" id=36870 trace_id=5638 func=ip_session_run_all_tuple line=4350 msg="SNAT 10.100.1.32->192.168.182.158:40828" id=36870 trace_id=5639 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=6, 192.168.183.1:80->192.168.182.158:40828) from port2." id=36870 trace_id=5639 func=resolve_ip_tuple_fast line=3405 msg="Find an existing session, id-00019c0b, reply direction" id=36870 trace_id=5639 func=ipv4_fast_cb line=56 msg="enter fast path" id=36870 trace_id=5639 func=ip_session_run_all_tuple line=4362 msg="DNAT 192.168.182.158:40828->10.100.1.32:61400" id=36870 trace_id=5640 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=47, 10.100.1.32:0->10.100.0.158:0) from lan." id=36870 trace_id=5640 func=wccp_gre_decap line=434 msg="feed to dev=port2 wccp=00000000" id=36870 trace_id=5640 func=__ip_session_run_tuple line=1830 msg="DNAT 192.168.182.158:55672->10.100.1.194:59868 " id=36870 trace_id=5641 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=6, 192.168.183.1:80->192.168.182.158:40828) from port2." id=36870 trace_id=5641 func=resolve_ip_tuple_fast line=3405 msg="Find an existing session, id-00019c0b, reply direction" id=36870 trace_id=5641 func=ipv4_fast_cb line=56 msg="enter fast path" id=36870 trace_id=5641 func=ip_session_run_all_tuple line=4362 msg="DNAT 192.168.182.158:40828->10.100.1.32:61400" id=36870 trace_id=5642 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=47, 10.100.1.32:0->10.100.0.158:0) from lan." id=36870 trace_id=5642 func=wccp_gre_decap line=434 msg="feed to dev=port2 wccp=00000000" id=36870 trace_id=5642 func=__ip_session_run_tuple line=1830 msg="DNAT 192.168.182.158:55672->10.100.1.194:59868" id=36870 trace_id=5643 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=6, 10.100.1.194:59868->192.168.183.1:80) from lan." id=36870 trace_id=5643 func=resolve_ip_tuple_fast line=3405 msg="Find an existing session, id-00019c0a, original direction" id=36870 trace_id=5643 func=__ip_session_run_tuple line=1816 msg="SNAT 10.100.1.194->192.168.182.158:55672" id=36870 trace_id=5643 func=wccp_output line=233 msg="gre_forward" id=36870 trace_id=5643 func=wccp_output line=275 msg="send packet via dev-lan" id=36870 trace_id=5644 func=resolve_ip_tuple_fast line=3371 msg="vd-root received a packet(proto=6, 10.100.1.32:61400->192.168.183.1:80) from lan." id=36870 trace_id=5644 func=resolve_ip_tuple_fast line=3405 msg="Find an existing session, id-00019c0b, original direction" id=36870 trace_id=5644 func=ipv4_fast_cb line=56 msg="enter fast path" id=36870 trace_id=5644 func=ip_session_run_all_tuple line=4350 msg="SNAT 10.100.1.32->192.168.182.158:40828" |
FG310B-4:
[2]: WCCP client receive packet from GRE tunnel, packet is natted and sent to Internet WCCP configuration between FortiGates (Router/Sever) and squid proxy using GRE tunneling: |
A pcap format sniffer trace captured on FG310B-4 showing a sample of traffic and wccp packets is attached as (310B-4-sample-sniff-port5.cap)