SSL Inspection - connection reset

[Dmp]

Hi everyone, I' ve been trying to figure out this issue for some time, i' m trying to implement SSL inspection for webfiltering and on some sites i' ve got connection resets while on others everything works beautifully. My main issue is that one of these sites is Google, and Facebook is another, each time i want to access this sites with SSL inspection, a connection reset ocurrs. A site that works, for example, www.ibm.com or support.fortinet.com. The CA certificate in the Fortigate was correctly imported in the client, also was signed by our internal root_ca, so no issues there. Several browsers tested (IE, Chrome, Firefox), Windows & Linux also tested. I attach a log with an attempt to access https://www.google.com. My system: Fortigate 60D with FortiOS 5.0.3. Any insight is welcome, thanks in advance.

[mbrowndcm]

That' s interesting that it works fine on some sites, but not all sites over https. Are you blocking some sites, and not others? You' re probably not blocking google.com, right? Try this instead:

 diag debug enable
 diag debug console timestamp enable
 diag debug flow show console enable
 diag debug flow show function-name enable
 diag debug flow filter addr 10.9.193.232
 diag debug flow filter port 443
 diag debug flow trace start 50
 

Then try from 10.9.193.232. To stop and clear:

 diag debug disable
 diag debug flow filter clear
 diag debug reset
 

Also try this:

 diag debug enable
 diag debug console timestamp enable
 diag debug app ssl -1
 

To stop and reset:

 diag debug disable
 diag debug reset
 

You may want to log printable output to a file by using puTTy. You might be seeing TCP RST because of your browser.

[Dmp]

Hi mbrowndcm, Thanks for the reply, you are correct, google is not filtered, what is more, even with webfilter not active (just SSL inspection, no AV, WF, etc) the connection still gets resetted. Attaching the log you requested, you' ve got 2 tries to get to www.google.com with flow trace first and another one with the ssl debugging at last. Regards.

[mbrowndcm]

 2013-06-14 15:12:06 69:ssl srv port close recv
 2013-06-14 15:12:06 69:shouldReset, type 1
 2013-06-14 15:12:06 [65]-<000.000000> [69]: 10.9.193.232:45210 --> 74.125.229.178:443: [SSL SVRRDRDY  ] Event - RESET_EVENT
 2013-06-14 15:12:06 [65]-<000.000000> [69]: 10.9.193.232:45210 --> 74.125.229.178:443: [SSL SVRRDRDY  ] RESET_STATE
 2013-06-14 15:12:06 [65]-[69] ips closed
 2013-06-14 15:12:06 ipsapp ses 69 close
 2013-06-14 15:12:06 ipsapp ses 69 send end msg 145 len 0 dir 0
 2013-06-14 15:12:06 69:checkConnectionEvent
 2013-06-14 15:12:06 69:resetConn
 2013-06-14 15:12:06 69:proxy_ssl_port_exit
 2013-06-14 15:12:06 69:proxy_ssl_port_close
 ...
 2013-06-14 15:12:09 70:checkConnectionEvent
 2013-06-14 15:12:09 check fd 12
 2013-06-14 15:12:09 clear client read
 2013-06-14 15:12:09 set client write
 2013-06-14 15:12:09 check fd 13
 2013-06-14 15:12:09 set server read
 2013-06-14 15:12:09 set server write
 2013-06-14 15:12:09 70:sslStateCheck
 2013-06-14 15:12:09 reset event
 2013-06-14 15:12:09 [65]-<000.000000> [70]: 10.9.193.232:45211 --> 74.125.229.178:443: [SSL LOOPEND   ] Event - RESET_EVENT
 2013-06-14 15:12:09 [65]-<000.000000> [70]: 10.9.193.232:45211 --> 74.125.229.178:443: [SSL LOOPEND   ] RESET_STATE
 2013-06-14 15:12:09 [65]-[70] ips closed
 2013-06-14 15:12:09 ipsapp ses 70 close
 2013-06-14 15:12:09 ipsapp ses 70 send end msg 147 len 0 dir 0
 2013-06-14 15:12:09 70:resetConn
 2013-06-14 15:12:09 70:proxy_ssl_port_exit
 2013-06-14 15:12:09 70:proxy_ssl_port_close
 

..seem interesting. Looks like your client is sending a RST. Sounds like a certificate error? What configuration method did you use? Did you install both the private key of the cert and the public cert to make the Fortigate a subordinate CA? " CA:TRUE" is listed in Certificate Details? [this was done by either using `openssl ca -extensions v3_ca` or by requesting a " subordinate CA" cert from a MSFT Windows CA] Sorry to review the trivialities.

[Dmp]

Hi, I think that if there was a certificate error all sites should fail and/or give an incorrect certificate error (i' ve made my share of trial and errors last year ). The certificate was created in the Fortigate itself (Certificates -> Local Certificates -> Generate), so there shouldn' t be any issues with the keys, the CSR was downloaded and sent to our root_CA for validation, then the validated certificate was uploaded, i' m attaching a screenshot of the certificate itself. Oh, and i' ve already tried generating the key+cert outside the FG, then importing it, still same result. I thought of the browser, already tried IE 10, Firefox 21 & Chrome 27, plus the test machine used for the traces i' ve sent is a brand new installed Linux Mint 15 with Firefox 20 and Chromium 25, all browsers same result, tried using SSL 3.0 only, again, the result is the same. Don' t be sorry for asking the trivialities, i usually make mistakes so no harm done in checking again. :) Thanks for the input, its really appreciated.

[mbrowndcm]

Cool. I agree, if it was the cert, it would fail globally (of course!). Have you tried testing with a total other client like `curl`?

curl --insecure https://google.com

That' s interesting, seeing:

 <HTML><HEAD><meta http-equiv=" content-type"  content=" text/html;charset=utf-8" >
 <TITLE>301 Moved</TITLE></HEAD><BODY>
 <H1>301 Moved</H1>
 The document has moved
 <A HREF=" https://www.google.com/" >here</A>.
 </BODY></HTML>
 

What happens if: 1) you create a URL Filter and exempt google.com? 2) you disable " Block HTTP Redirects by Rating" ? (I know this won' t do anything) 3) you access another 301 redirected page? 4) Try to review the logs to see if web filter or app control are dropping the packets... what about other layers in your packet flow beyond the ssl mitm proxy and the firewall policies? 5) If this was working before, but now is not, you can also try to restart the ssl daemon:

 diag test app ssl 99
 

[Dmp]

Let' s start with the first (and end with the last). a - Our good friend curl gives error 35, unknown SSL protocol error in connection with www.google.com:443. 1 - For tests sake i enabled WF and exempted google.com (it wasn' t enabled in the traces before), it gets a little better since now the redirect happens, but still after that the connection gets resetted (i' ve also already exempted my local google site). *Edit: Correct that, now it works... randomly, sometimes it resets, sometimes it works, let' s say works 85% of the time* 2 - It wasn' t enabled to begin with, so you' re right, it doesn' t do anything. :) 3 - See comment 1, yup. 4 - No packets dropped in the logs, still, please take into account that this reset happens just by adding SSL inspection, no other modules are needed for the connection drop to happen (i' ve tried to simplify things, it fails with or without the other modules running); so the only things enabled ATM are Vdoms, SSL inspection and 1 firewall policy, that' s it. 5 - It never worked, still, for tests sake webfilter (global) # diagnose test application sslacceptor 99 2013-06-14 17:27:42 restarting ssl proxy webfilter (global) # diagnose test application sslworker 99 SSL Worker 0: 2013-06-14 17:27:50 restarting ssl proxy webfilter (global) #

[mbrowndcm]

Cool. I had a problem yesterday where all my HTTPS inspection... just... dropped... isolated in a single VDOM. wat. I restarted the `ssl` daemon and it worked. So you mean you simply have " Enable Deep Scanning" enabled in the protocol options object, don' t have HTTPS Deep Scanning enabled in a web filter, and no other UTM policies attached to the policy, and you are facing this state? That' s crazy, but I have no other idea. With my problem above, I quickly reverted to a _web filter_ I had with no HTTPS Deep Scanning enabled, and traffic passed through HTTPS (even with a protocol options object assigned with Enable Deep Scanning enabled!). I' d just call support.

[Dmp]

Yes, i understand, i have the advantage that this is a brand new 60D so i can reboot it whenever needed. That' s what i mean, in FortiOS 5 " terms" , 1 policy (internal1 -> wan1, ALL to ALL protocols accepted), in " Security Profiles" just SSL Inspection enabled and no other funky stuff apart from the custom certificate, all this being contained in root vdom. I' ve unpacked the 60D 3 days ago and factory resetted, upgraded to 5.0.3, just created the vdoms and that' s it from the " hardware" POV. I don' t think that issue lies in the unit, but i' ll try with a brand new one on Monday and will also try to get a new MPLS line with my TelCo, all this is beyond weird.. Anyway, as said before, i highly appreciate your feedback on the matter, thank you very much.

[Dmp]

Just a little follow-up, i' ve replaced the unit and still the same error, downgraded FortiOS to 5.01 (the lowest i can go on 60D) and still the same. Weirdly enough, i' ve tested it with one of my 60C and everything works beautifully, so either its FortiOS5 having a nasty behaviour or ...