So you want to handle the text from a proxy or other app which has deflate'd the content?

Setup a log with debugging info

The first most valuable thing for diagnosing any issues is to create a compression log which will show you the Accept-Encoding (AE) permitted by the client (browser) along with the Transfer-Encoding (TE) and Content-Encoding (CE) for the request body with the total input byte count. We'll prefix each entry with the client ip/host and their request line. And the meat of the response to the client are the response size (bytes transmitted), response Transfer-Encoding and response Content-Encoding. Throughout these examples we'll be sharing the results as gathered with the following config snippet;

LogFormat "%h \"%r\" %>s %b (%O) TE:\"%{Transfer-Encoding}o\" CE:\"%{Content-Encoding}o\" <-- %I AE:\"%{Accept-Encoding}i\" TE:\"%{Transfer-Encoding}i\" CE:\"%{Content-Encoding}i\"" compression

CustomLog logs/compression.log compression

Configuring a local test server

The first thing is to provide a server to test against which deflates it's output. We can do this in the same instance of Apache, simply set up an alias to the htdocs for /deflated/
requests for the very same documents in /.

Alias /deflated "/path-to/htdocs"
<Location "/deflated">
    SetOutputFilter DEFLATE
</Location>

Requesting the same LICENSE.txt from both / and /deflated/ reveals the impact deflate can offer;

127.0.0.1 "GET /LICENSE.txt HTTP/1.1" 200 39858 (40146) TE:"-" CE:"-" <-- 412 AE:"gzip,deflate" TE:"-" CE:"-" 
127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 418 AE:"gzip,deflate" TE:"-" CE:"-"

Note the response body has shrunk nearly to a quarter of it's original size.

Proxy requests to the local test server

For our first experiment (and we will keep it in our config as the LAST snippet, because this would override any other example below if it appears first), we will create a reverse-proxy (gateway) to our own server's content, everything requested from /proxy/ will actually be proxied from / of our server.

ProxyPass /proxy/ http://127.0.0.1/
ProxyPassReverse /proxy/ http://127.0.0.1/

The compression.log gets interesting here. The first request to be satisfied is our back-end proxied request from ourself, once it's completed the /proxy/ request can be completed. In this case, the responses are identical, we aren't manipulating the Content-Encoding as the request passes through the proxy;

127.0.0.1 "GET /LICENSE.txt HTTP/1.1" 200 39858 (40146) TE:"-" CE:"-" <-- 501 AE:"gzip,deflate" TE:"-" CE:"-" 
127.0.0.1 "GET /proxy/LICENSE.txt HTTP/1.1" 200 39858 (40146) TE:"-" CE:"-" <-- 418 AE:"gzip,deflate" TE:"-" CE:"-" 

127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 510 AE:"gzip,deflate" TE:"-" CE:"-" 
127.0.0.1 "GET /proxy/deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 427 AE:"gzip,deflate" TE:"-" CE:"-"

Remove the request for compression

Now the quick method of disabling compression is to never request a compressed document from the back-end machine, which we can accomplish by unset'ting the Accept-Encoding header from the client.

ProxyPass /proxy/nodeflate/ http://127.0.0.1/deflated/
<Location "/proxy/nodeflate/">
    RequestHeader unset Accept-Encoding
</Location>

And lo, neither the backend or proxy request appears to support gzip encoding;

127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 39858 (40169) TE:"-" CE:"-" <-- 479 AE:"-" TE:"-" CE:"-" 
127.0.0.1 "GET /proxy/nodeflate/LICENSE.txt HTTP/1.1" 200 39858 (40169) TE:"-" CE:"-" <-- 428 AE:"-" TE:"-" CE:"-"

Keep compression but inflate at the proxy

But if we want to continue to accept deflated content from the back end server for bandwidth conservation, it's possible to use mod_deflate to perform response re-inflation. We only want to do this if the back end server provided a response in Content-Encoding:gzip, or else we will be trying to unpack an apparently corrupt data stream;

FilterProvider gzinflate INFLATE resp=Content-Encoding $gzip

ProxyPass /proxy/reinflate/ http://127.0.0.1/deflated/
<Location "/proxy/reinflate/">
    FilterChain gzinflate
</Location>

Now when a request is made through /proxy/reinflate/, we will take the deflated back end content and explode it. Note that the (2nd) response to the client is nearly four times larger, again, than the backend request;

127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 510 AE:"gzip,deflate" TE:"-" CE:"-" 
127.0.0.1 "GET /proxy/reinflate/LICENSE.txt HTTP/1.1" 200 39858 (40219) TE:"chunked" CE:"-" <-- 428 AE:"gzip,deflate" TE:"-" CE:"-"

Transform the inflated content

Let's take this one step further, and add on a transformation. There's an experimental filter mod_case_filter which upper-cases the response, and it's perfect for illustrating the transformation. We can't upper-case the deflated binary stream, so we'll have to inflate it, then transform it.

NOTE A one line bug in mod_case_filter.c that must be fixed in 2.2.6 Insert the line;

        apr_brigade_cleanup(pbbIn);

immediately before this line;

        return ap_pass_brigade(f->next, pbbOut);

and rebuild the module.

So using the example above, we'll build on it with another FilterProvider we'll name ucase, and apply both gzinflate and ucase in that order to all /proxy/ucase/ requests;

FilterProvider gzinflate INFLATE resp=Content-Encoding $gzip
FilterProvider ucase CaseFilter Content-Type $text/

ProxyPass /proxy/ucase/ http://127.0.0.1/deflated/
<Location "/proxy/ucase/">
    FilterChain +gzinflate +ucase
</Location>

And the resulting log shows that the response to the client remains undeflated, and if you inspect it in the browser, it's irritatingly in all caps;

127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 513 AE:"gzip,deflate" TE:"-" CE:"-"
127.0.0.1 "GET /proxy/ucase/LICENSE.txt HTTP/1.1" 200 39858 (40219) TE:"chunked" CE:"-" <-- 427 AE:"gzip,deflate" TE:"-" CE:"-"

Redeflate the transformed content

Building on those two examples, we'll take this one step further, and ensure the final response can in fact be deflated to the client, so we'll name our redeflate filter gzdeflate and apply all three filters, in specific order, to inflate, manipulate, and re-deflate the response;

FilterProvider gzinflate INFLATE resp=Content-Encoding $gzip
FilterProvider ucase CaseFilter Content-Type $text/
FilterProvider gzdeflate DEFLATE Content-Type $text/

ProxyPass /proxy/ucase/ http://127.0.0.1/deflated/
<Location "/proxy/ucase/">
    FilterChain +gzinflate +ucase +gzdeflate
</Location>

Note a fun side effect on deflation of upper-casing the document;

127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 514 AE:"gzip,deflate" TE:"-" CE:"-"
127.0.0.1 "GET /proxy/ucase/LICENSE.txt HTTP/1.1" 200 9600 (9934) TE:"-" CE:"gzip" <-- 428 AE:"gzip,deflate" TE:"-" CE:"-"

With only 26 alpha characters rather than 52 (upper and lower case) it compresses down more efficiently than the back end had provided it to us.

Forward proxies

We can actually do this for forward proxies as well. Here is an example proxy server which inflating every C-E:gzip compressed response and serves only plain text to the clients using our proxy, we save about 10%.

LogFormat "%h \"%r\" %>s %b (%O) TE:\"%{Transfer-Encoding}o\" CE:\"%{Content-Encoding}o\" <-- %I AE:\"%{Accept-Encoding}i\" TE:\"%{Transfer-Encoding}i\" CE:\"%{Content-Encoding}i\"" compression

FilterProvider gzinflate INFLATE resp=Content-Encoding $gzip

<VirtualHost 192.168.0.254:80>
    ErrorLog logs/proxyerror.log
    CustomLog logs/proxyaccess.log common
    CustomLog logs/proxycompression.log compression
    ProxyRequests On
    ProxyVia On

    <Proxy http:*>
        FilterChain gzinflate
        Order deny,allow
        Deny from all
        Allow from 127. 10. 192.168.
    </Proxy>
</VirtualHost>
  • No labels