Technical Tip: How to troubleshoot Static URL Filter in proxy mode using 'wad debug'

[FrankY1]

Description	This article explains the process for troubleshooting the static URL filter in proxy mode by running 'wad debug' commands.
Scope	FortiOS.
Solution	Debug commands: diagnose debug reset diagnose wad debug enable category http diagnose wad debug enable category http2 diagnose wad debug enable category ssl diagnose wad debug enable category policy diagnose wad debug enable category scan diagnose wad debug enable level verbose diagnose wad filter src <source-ip> diagnose debug enable In the example below, a webfilter profile with the name 'test' is created, with a static URL filter that exempts 'www.google.com'. The static URL filter's ID is found to be '11'. Static URL configuration: config webfilter urlfilter edit 11     set name "Auto-webfilter-urlfilter_gd1vfip7s"     config entries         ...         edit 4             set url "www.google.com"         next      end end config webfilter profile    edit "test"       set feature-set proxy       config web           set urlfilter-table 12    end Debug output and analysis: Search for 'www.google.com'" and locate the matching process ID, socket ID, and request ID. Those numbers can be used to identify the most relevant outputs in the URL matching process.  [V][p:2240][s:15143][r:1278] wad_http_req_alloc :1753 req=0x7ff6fe8f1608(ses_ctx:t|Phx|Me|Hfe|C|A1|O) dst(orig=172.217.24.36:443 dst=172.217.24.36:443 srv=172.217.24.36:443) hs=0x7ff701dedd48 clt_port=0x7ff6fe937d38 svr_port=0x7ff6fe9397b0 [I][p:2240][s:15143][r:1278] wad_dump_http_request :2842 hreq=0x7ff6fe8f1608 Received request from client: 10.16.2.229:50927 [V][p:2240][s:15143][r:1278] wad_http_marker_uri :1270 path=/ len=1 [V][p:2240][s:15143][r:1278] wad_http_parse_host :1649 host_len=14 [I][p:2240][s:15143][r:1278] wad_http_parse_host :1681 host=[14]www.google.com [I][p:2240][s:15143][r:1278] wad_http_str_canonicalize :2196 enc=0 path=/ len=1 changes=0 [V][p:2240][s:15143][r:1278] wad_http_normalize_uri :2513 host_len=14 path_len=1 query_len=0 [I][p:2240][s:15143][r:1278] wad_http_req_detect_special :15972 captive_portal detected: false, preflight=(null) [V][p:2240][s:15143][r:1278] wad_http_req_exec_act :14365 request(0x7ff6fe8f1608), intercept(pass), block(0) [V][p:2240][s:15143][r:1278] wad_http_req_exec_act :14454 dst_addr_type=1 wc_nontp=0 sec_web=1 web_cache=0 req_bypass=1 [V][p:2240][s:15143][r:1278] wad_http_req_check_policy :12937 start match policy vd=1(ses_ctx:t|Phx|Me|Hfe|C|A1|O) (10.56.241.75:50927@11->172.217.24.36:443@12) absUrl=0 [V][p:2240][s:15143][r:1278] wad_http_req_check_policy :12973 same policy take shortcut [I][p:2240][s:15143][r:1278] wad_http_req_proc_policy :10736 ses_ctx:t|Phx|Me|Hfe|C|A1|O conn_srv=0 fwd_srv=<nil> [I][p:2240][s:15143][r:1278] wad_http_req_proc_policy :10855 policy result:vf_id=1:0 sec_profile=0x7ff6ffe7e248 set_cookie=0 [I][p:2240][s:15143][r:1278] wad_http_urlfilter_check :386 uri_norm=1 inval_host=0 inval_url=0 scan-hdr/body=1/0 url local=1 block=0 user-cat=1 allow=0 ftgd=1 keyword=0 wisp=0 [I][p:2240][s:15143][r:1278] wad_url_filter_req_alloc :655 url_req=0x7ff6ffe96ba0 id=0 [I][p:2240][s:15143][r:1278] wad_http_urlfilter_check :470 URL filter ret=0 url local=1 block=0 user-cat=1 allow=0 ftgd=1 [I][p:2240][s:15143][r:1278] wad_ssl_port_caps_on_task :13790 wsp=0x7ff6fe937b08/6 cts 2 pts 3 hs 34/28 cpcs 0 ppcs 0 se 0 ed 0/0 ph 0 pti 0/0/32 cti 0/0 ci 1/39/5 cto 0/0 wb 0/0 [I][p:2240][s:15143][r:1278] wad_ssl_port_caps_on_task :13790 wsp=0x7ff6fe937b08/6 cts 3 pts 3 hs 34/28 cpcs 0 ppcs 0 se 0 ed 0/0 ph 0 pti 0/0/32 cti 0/0 ci 1/39/39 cto 0/0 wb 0/0 [I][p:2240][s:15143][r:1278] wad_ssl_port_caps_on_task :13790 wsp=0x7ff6fe937b08/6 cts 2 pts 3 hs 34/28 cpcs 0 ppcs 0 se 0 ed 0/0 ph 0 pti 0/0/32 cti 1/39 ci 0/0/5 cto 0/0 wb 0/0 [V][p:2240][s:15143][r:1278] wad_ssl_port_caps_on_dec_start :17116 sp=0x7ff6fe937b08/6 dec start 0x7ff700635700 type 23 ver 0303 len 34 (e8 19 16 6f 25 71 e0 4c 8b 10 1f 96 04 ff 00 65 43 52 88 3b 50 57 ca 88 26 a9 6f 39 2e d7 95 43 79 49 ) [V][p:2240][s:15143][r:1278] wad_ssl_port_caps_on_dec_done :17142 sp=0x7ff6fe937b08/6 dec done 0x7ff700635700 type 23 ver 0303 status 0 len 17 (00 00 08 06 00 00 00 00 00 00 00 00 00 00 00 00 01 ) [I][p:2240][s:15143][r:1278] wad_ssl_port_caps_on_task :13790 wsp=0x7ff6fe937b08/6 cts 2 pts 3 hs 34/28 cpcs 0 ppcs 0 se 0 ed 0/0 ph 0 pti 1/17/32 cti 0/0 ci 0/0/5 cto 0/0 wb 0/0 [I][p:2240][s:15143][r:1278] wad_ssl_app_port_fts_in_write :19421 sp=0x7ff6fe937b08/6 decrypted up stream len=17 [V][p:2240][s:15143][r:1278] wad_ssl_app_port_out_ops_read_buff:19811 sp=0x7ff6fe939580/7 plain down stream len=17 [I][p:2240][s:15143][r:1278] wad_ssl_port_caps_on_task :13790 wsp=0x7ff6fe937b08/6 cts 2 pts 3 hs 34/28 cpcs 0 ppcs 0 se 0 ed 0/0 ph 0 pti 0/0/32 cti 0/0 ci 0/0/5 cto 0/0 wb 0/0 [I][p:2240][s:15143][r:1278] wad_http_url_filter_check_local :3487 hreq=0x7ff6fe8f1608 prof=test host=www.google.com(172.217.24.36) vd=FG-traffic id=0 rate=0 [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_hs :2261 url1=www.google.com url2=/ [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_cb :2236 hs_id=15 0->14 flag=0 ent: type=0 vd=1 tab=1 id=15 match=0 [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_cb :2236 hs_id=243 0->14 flag=0 ent: type=0 vd=1 tab=2 id=15 match=0 [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_cb :2236 hs_id=258 0->14 flag=0 ent: type=0 vd=1 tab=3 id=15 match=0 [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_cb :2236 hs_id=483 0->14 flag=0 ent: type=0 vd=1 tab=6 id=15 match=0 [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_cb :2236 hs_id=695 0->14 flag=0 ent: type=0 vd=1 tab=7 id=2 match=0 [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_cb :2236 hs_id=711 0->14 flag=0 ent: type=0 vd=1 tab=8 id=15 match=0 [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_cb :2236 hs_id=936 0->14 flag=0 ent: type=0 vd=1 tab=9 id=15 match=0 [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_cb :2236 hs_id=1147 0->14 flag=0 ent: type=0 vd=1 tab=10 id=1 match=0 [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_cb :2236 hs_id=694 0->14 flag=0 ent: type=0 vd=1 tab=7 id=1 match=0 [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_cb :2236 hs_id=1152 0->14 flag=0 ent: type=0 vd=1 tab=11 id=4 match=1 <-- This indicates a match is found, with vdom ID 1, URL filter ID 11, and URL entry ID 4.  [V][p:2240][s:15143][r:1278] wad_urlfilter_local_lookup_hs :2261 url1=172.217.24.36 url2=/ [I][p:2240][s:15143][r:1278] wad_url_filter_local_proc_result :2504 hreq=0x7ff6fe8f1608 wfp=0x7ff701ec9510 ssl_url_chk=1 exempt: exempt: AV| CONTENT| JAVA_COOKIE| DLP| FTGD| RANGEBLOCK| ANTIPHISH|end. [I][p:2240][s:15143][r:1278] wad_url_filter_local_request :901 hreq=0x7ff6fe8f1608 wfp=0x7ff701ec9510 ssl_url_chk=0 acion=allow: >> X-FTGD(x-ssl=0 kwd=0 per_usr_bal/log=0/0) [V][p:2240][s:15143][r:1278] wad_url_filter_log_result :2436 url_req=0x7ff6ffe96ba0 result=0 log=1 type=exempt lasttype=exempt [V][p:2240][s:15143][r:1278] wad_url_filter_log :2196 http req=0x7ff6fe8f1608 action=0 type=exempt keyword=0/-1 search_log=0/0 [I][p:2240][s:15143][r:1278] wad_url_filter_cancel :669 type=0 req=0x7ff6fe8f1608 url_req=0x7ff6ffe96ba0 id=0 [I][p:2240][s:15143][r:1278] wad_http_req_proc_waf :1375 req=0x7ff6fe8f1608 ssl.deep_scan=1 proto=10 exempt=191 waf=(nil) body_len=18446744073709551615 ua=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 Edg/126.0.0.0 skip_scan=0 [I][p:2240][s:15143][r:1278] wad_http_srv_attach_req :813 [0x7ff6fe8f1608] Use old server0x7ff7026545d8: :0 [V][p:2240][s:15143][r:1278] wad_http_req_get_svr :9427 http session 0x7ff701dedd48 req=0x7ff6fe8f1608 connected [V][p:2240][s:15143][r:1278] wad_http_msg_start_setup_proc :2225 msg(0x7ff6fe8f1608) proc-setup started from: req_casb. [V][p:2240][s:15143][r:1278] wad_http_def_proc_msg_plan :2187 msg(0x7ff6fe8f1608) setting up processor(req_casb) [V][p:2240][s:15143][r:1278] wad_http_def_proc_msg_plan :2187 msg(0x7ff6fe8f1608) setting up processor(req_scan) [W][p:2240][s:15143][r:1278] wad_req_setup_scan_proc :1832 Skipped [V][p:2240][s:15143][r:1278] wad_http_def_proc_msg_plan :2187 msg(0x7ff6fe8f1608) setting up processor(req_quota) [V][p:2240][s:15143][r:1278] wad_http_def_proc_msg_plan :2187 msg(0x7ff6fe8f1608) setting up processor(req_cache) [V][p:2240][s:15143][r:1278] wad_http_def_proc_msg_plan :2187 msg(0x7ff6fe8f1608) setting up processor(req_vs) [V][p:2240][s:15143][r:1278] wad_http_def_proc_msg_plan :2187 msg(0x7ff6fe8f1608) setting up processor(req_forward) [I][p:2240][s:15143][r:1278] wad_dump_fwd_http_req :2856 hreq=0x7ff6fe8f1608 Forward request to server: [V][p:2240][s:15143][r:1278] wad_hpack_enc_req :1822 path='/' [V][p:2240][s:15143][r:1278] wad_ssl_app_port_out_ops_read_buff:19811 sp=0x7ff6fe939580/7 plain down stream len=1383