Still about chunked encoding: can anyon translate what the faq says?

[luis_abreu]

Hello guys,

Even though I'm still not sure on how FortiWeb  supports chunked encoding, I'm hoping that someone can help me understand the faq's content on this topic, specificaly on the behavior form 7.0.2 onwards. The doc starts by saying the following:

From 7.0.2, FortiWeb replaced set chunk decoding enable/disable with set chunk encoding disable/enable.

• The default configuration is disabled, which equals to set chunk decoding enable in 7.0.1; FortiWeb will decode chunked response and convert it with Content-Length.

• When configured as set chunk encoding enable on 7.0.2, FortiWeb decodes and reassembles the chunked response, performs the WAF modules’ operations, and encodes the new content with chunked again, then sends it to the clients.

I understood it as saying that whenever FortiWeb detects a chunked encoded response from an HTTP 1.1 server, it will receive all the packets from the server, decode it and then apply all the specified security modules of the Web Protection Profile to the assembled buffered message before it is returned to the client with the chunked header appended (ie, it returns the message as single chunk without the Content-Lenght header).

However, that some doc then goes on and says this:

From 7.0.2, when set chunk encoding enable, instead of delaying sending packets to the client until all content is available, the server will:

• Send the response in chunks.

• Add a Transfer-Encoding: chunked header to the chunks.

• Apply markers within the content to indicate the length of each chunk and whether that particular chunk is the last chunk that the server is sending.

• Under some conditions, chunk decoding module will not take action:

  • No web protection profile is bound to a server policy;

  • No modules enabled in a web protection profile;

  • Modules that depend on chunk decoded data are not enabled in web protection profile (e.g. compress, xml validation);

  • When chunked response size exceeds max-cache-size, FortiWeb will not decode chunked content.

This is completely different from the previous description. According to this,  FortiWeb is able to "decode" each chunk it gets from the server, pass it through the security modules before sending each of those chunks to the client. The problem with these descriptions is that (to me) they  describe conflicting behaviors  applied from 7.0.2 onwards and that doesn't make sense.

So, which of these descriptions is correct? Anyone?

Thanks.

[sjoshi]

From version 7.0.2 onwards, FortiWeb's behavior with chunked encoding involves decoding and reassembling the chunked response, applying the WAF modules' operations to the assembled message, and then encoding the new content with chunked before sending it to the clients. Additionally, when chunk encoding is enabled, FortiWeb will send the response in chunks, add a Transfer-Encoding: chunked header to each chunk, and apply markers within the content to indicate the length of each chunk and whether it is the last one. However, the chunk decoding module will not take action under certain conditions such as when no web protection profile is bound to a server policy, no modules are enabled in a web protection profile, or when the chunked response size exceeds the max-cache-size.

[luis_abreu]

Hello again.

Once more, thanks for your reply.

---

@sjoshi wrote:

From version 7.0.2 onwards, FortiWeb's behavior with chunked encoding involves decoding and reassembling the chunked response, applying the WAF modules' operations to the assembled message, and then encoding the new content with chunked before sending it to the clients. Additionally, when chunk encoding is enabled, FortiWeb

I'm sorry, but I'm having a hard time seeing that in the docs. For instance, what you describe will only happen when the server has the set chunk encoding enable (look at the 1st quote I've shown on my post for the From 7.0.2 onward). Ie, what you describe won't happen by default (ie, when the FortiWeb has chunked module disabled) according to the snippets I've sent you before. 

Btw, what I've seen in the tests I've run in our test environment (#Fortiweb 7.2.X) is that the first description is the one that is correct. What I'd really like to know is from what version onward does FortiWeb  apply the second description (the one that mentions that packets aren't delayed in order to be reassembled by FortiWeb).

And BTW I still don't see anything on the docs that explain  why enabling the chunked encoding on the FortiWeb transforms all requests returned by the real server into chunked responses...

Thanks again.