Questions on "diagnose test application urlfilter"

[AlexFeren]

Questions to those in the know:

1. In the printout: FG60C (global) # diagnose test application urlfilter 3 Saving to file [/tmp/urcCache.txt] Cache Contents: -=-=-=-=-=-=-=- Cache Mode:   TTL Cache DB Ver: 17.53220 Domain  |IP       DB Ver T URL 1e000000|00000000 17.53219 P B[link]https://vuexcasht3.ad.vu.edu.au/[/link] : what does the "T" indicator show and what values (such a "P", above) can be shown? Also, what does "B" prepending the URL show?

2. is it possible to (regexp or otherwise) filter printout generated by "diagnose test application urlfilter 3" for specific URL? (Currently, I'm forced to use grep on /tmp/urcCache.txt).

3. What is the difference between "diagnose test application urlfilter 3" and "diagnose test application urlfilter 16"

4. When I issue the command, I get an incomplete printout, eg. FG60C (global) # diagnose test application urlfilter 16 Saving to file [/tmp/urcCache.txt] Cache Contents: -=-=-=-=-=-=-=- Cache Mode:   TTL Cache DB Ver: 17.53277 Domain  |IP       DB Ver T URL 34000000|00000000 17.53275 P Bhttps://lastpass.com/ 34000000|34000000 17.53275 P Bhttps://sync.xmarks.com/ 1e000000|00000000 17.53275 P Bhttps://foo-border-faz-3000e.net.vu.edu.au/ 34000000|38000000 17.53275 P Bhttp://realtime.services.disqus.com/ 34000000|38000000 17.53274 P Bhttp://fortinetdocument.disqus.com/ 1d000000|00000000 17.53274 P Bhttp://fast.fonts.net/ 34000000|34000000 17.53274 P Bhttps://support.fortinet.com/ 4c0000   Why is the printout incomplete/cut-off/curtailed?

R's, Alex

[neonbit]

Not sure on the prefixes. There's a wiki article on the differences between '3' and '16' (not very descriptive) as per below:

3 display WF cache contents

16 display WF cache contents of prefix type

http://wiki.diagnose.fort..._application_urlfilter

[scao_FTNT]

I got update from FOS team

##############

1. In the printout: FG60C (global) # diagnose test application urlfilter 3 Saving to file [/tmp/urcCache.txt] Cache Contents: -=-=-=-=-=-=-=- Cache Mode:   TTL Cache DB Ver: 17.53220 Domain  |IP       DB Ver T URL 1e000000|00000000 17.53219 P B[link]https://vuexcasht3.ad.vu.edu.au/[/link] : what does the "T" indicator show and what values (such a "P", above) can be shown? Also, what does "B" prepending the URL show?   T=Type, URL Match Type: P  Prefix match; E  Exact Match;   B shows URL Source, it correlate to urlfilter debug URL Source:-- Source=0=A,Unknown,  1=B,HTTP Header,  2=C, SNI Name, 3=D, Server Certificate CN Name      Example -- diag test application urlfilter 3 Saving to file [/tmp/urcCache.txt] Cache Contents: -=-=-=-=-=-=-=- Cache Mode:   TTL Cache DB Ver: 17.54961 Domain  |IP       DB Ver T URL 34000000|34000000 17.54961 P B[link]http://www.fortinet.com/[/link] 34000000|00000000 17.54961 E Bhttp://www.cisco.com/c/en...es/order-services.html           2. is it possible to (regexp or otherwise) filter printout generated by "diagnose test application urlfilter 3" for specific URL? (Currently, I'm forced to use grep on /tmp/urcCache.txt).   --Currently NOT Supported     3. What is the difference between "diagnose test application urlfilter 3" and "diagnose test application urlfilter 16"   As explained in question 1, it print out different prefix type     4. When I issue the command, I get an incomplete printout, eg. FG60C (global) # diagnose test application urlfilter 16   Saving to file [/tmp/urcCache.txt]   Cache Contents: -=-=-=-=-=-=-=- Cache Mode:   TTL Cache DB Ver: 17.53277   Domain  |IP       DB Ver T URL 34000000|00000000 17.53275 P B[link]https://lastpass.com/[/link] 34000000|34000000 17.53275 P B[link]https://sync.xmarks.com/[/link] 1e000000|00000000 17.53275 P B[link]https://foo-border-faz-3000e.net.vu.edu.au/[/link] 34000000|38000000 17.53275 P B[link]http://realtime.services.disqus.com/[/link] 34000000|38000000 17.53274 P B[link]http://fortinetdocument.disqus.com/[/link] 1d000000|00000000 17.53274 P B[link]http://fast.fonts.net/[/link] 34000000|34000000 17.53274 P B[link]https://support.fortinet.com/[/link] 4c0000   Why is the printout incomplete/cut-off/curtailed?     This might caused by console output cache full.     #########   Thanks   Simon

[AlexFeren]

Hi Simon, thank for answering....

> B shows URL Source, it correlate to urlfilter debug URL Source:--

can you please be more specific? Are you referring to "diagnose debug urlfilter test-url" ?

> This might caused by console output cache full.

So, how can I get the complete printout?

R's, Alex

[scao_FTNT · Accepted solution]

I got another update, sorry for the late response

##############

"B" present how the FGT get/extract the URL from(url source), other posibility can be: A =Unknown B =HTTP Header C =SNI Name D =Server's Certificate CN Name For example, you use HTTPS to visit some site, FGT can extract URL from C, D, or B, it depend on your policy configuration: If you apply certificate inspection ssl-ssh-profile to your policy, FGT is likely to get URL from SNI Name; If  you apply deep inspection ssl-ssh-profile to your policy, FGT will exact URL from HTTP Header. For HTTP, it can be from HTTP header only.   command line #diagnose debug urlfilter <test-url> is used for test your url rating result on FGT.   >So, how can I get the complete printout? you can check the disk file /tmp/urcCache.txt   Thanks   Simon

[AlexFeren]

Hi Simon,

> you can check the disk file /tmp/urcCache.txt

'fnsysctl' is unsupported!

> command line #diagnose debug urlfilter <test-url> is used for test your url rating result on FGT.

Sorry to change direction, but how would I use 'diagnose debug urlfilter test-url'? Observe:

FG60C (global) # get system fortiguard 

:

webfilter-force-off : disable webfilter-cache     : enable webfilter-cache-ttl : 3600 webfilter-license   : Contract webfilter-expiration: Mon Aug  1 2016 webfilter-timeout   : 15 webfilter-sdns-server-ip: webfilter-sdns-server-port: 53 source-ip           : 0.0.0.0 ddns-server-ip      : 0.0.0.0 ddns-server-port    : 443

FG60C (root) # show webfilter urlfilter config webfilter urlfilter     edit 1         set name "Block_Ads_Security_WF"             config entries                 edit 1                     set url "s.yimg.com/gs/apex/mediastore/*"                     set type wildcard                     set action block                 next             end     next end

FG60C (root) # diagnose debug info debug output:           enable console timestamp:      disable console no user log message:    disable urlfilter debug level:  -1 (0xffffffff) CLI debug level:        3 FG60C (root) # diagnose debug urlfilter test-url s.yimg.com/gs/apex/mediastore/alex Not found in cache

What does 'Not found in cache' mean and what is correct method to use the command?

R's, Alex

[AlexFeren]

Hi experts,

I wondering, if 4 years later, anyone can answer my previous question: "how would I use 'diagnose debug urlfilter test-url'; and, what does updated response, "URL test cache miss" means (when "diagnose test application urlfilter 3" is showing test site in WF Cache)?

R's, Alex

[AlexFeren]

Finally.. Bug-id #553593.