http compression with gzip

section31 · 10-02-2004, 10:29 PM

This is for those who are famliiar with compressing webpages on the fly with apache server.
Is there a local tool I can use on a linux box to check the compression for websites. I know there are several sites that offer these tools like this one -> http://www.desilva.biz/gzip-test.php
But I want to know how i can check myself or perhaps create a script like his from scratch. Maybe a flag I missed for wget or something similar? Anyone?

**greffov** · 10-03-2004, 04:28 AM

ah, that's simple.

either telnet to the webserver, so you can see the headers (including gzip compression flag) or write some script to do so.

you can do it simply with lynx

Code:

lynx -head http://www.pctechtalk.com/forums/

which will result in something like:

Code:

HTTP/1.1 200 OK
Date: Sun, 03 Oct 2004 08:27:38 GMT
Server: Apache/2.0.46 (Red Hat)
Accept-Ranges: bytes
X-Powered-By: PHP/4.3.2
Set-Cookie: ; path=/; domain=.pctec
htalk.com
Set-Cookie: bblastvisit=1096792058; expires=Mon, 03-Oct-2005 08:27:38 GMT; path
=/; domain=.pctechtalk.com
Content-Encoding: gzip
Connection: close
Content-Type: text/html

section31 · 10-03-2004, 05:02 AM

hm...for some reason out of all the forums ii use..yahoo always throws emails from this forum to the bulk mail...hence taking me so long to reply to this...sorry

hm...thats just to find the headers, notice on that site, he also is able to find out how many bytes it is compressed/uncompressed.

**greffov** · 10-03-2004, 05:49 AM

yeah, well, that's simple. They retrieve the whole site, then gunzip it and calculate the ratio.

If you really want I can make a PHP site that does the same as theirs. You can do it easily with Java as well.

I only expect this functionality is only needed a few times so not worth the coding...

section31 · 10-03-2004, 08:57 AM

Quote:

Originally posted by greffov
yeah, well, that's simple. They retrieve the whole site, then gunzip it and calculate the ratio.

If you really want I can make a PHP site that does the same as theirs. You can do it easily with Java as well.

I only expect this functionality is only needed a few times so not worth the coding...

nope, that crossed my mind...but thats not what he does.. U see there are several compression modes for gzip. 1-9 i believe. So he does something else. I actually posted on his forums, to see if he replies...so far nothing...I'm hoping he will give me his source if thats possible. If not an alternative would be nice.

**greffov** · 10-03-2004, 10:46 AM

You can never predict the compression rate of a zip algorithm. (g)Zip works by existance of duplicates of data, if none exist, the zip algorithm will not be able to be of any use. The level (1-9) affects the greedyness of the algorithm, which can result in a better compression ratio, but takes more CPU and memory.

The only way to calculate a compression ratio is to compare the compressed and uncompressed data.

section31 · 10-03-2004, 01:22 PM

I see....so basically what you're saying is that, the site just gives an estimation of what it is compressed to. Since there is no way to predict what compression mode the server is using.

**greffov** · 10-03-2004, 01:26 PM

I think the site is just retrieving the website. It's not that hard, and it enables calculation of the compression rate.

The rates differ for me for several pages from the same website, so that makes me feel they really don't estimate anything, but just calculate the value.

10-02-2004, 10:29 PM	#1 (permalink)
section31 Registered User Join Date: Sep 2004 Posts: 35	http compression with gzip This is for those who are famliiar with compressing webpages on the fly with apache server. Is there a local tool I can use on a linux box to check the compression for websites. I know there are several sites that offer these tools like this one -> http://www.desilva.biz/gzip-test.php But I want to know how i can check myself or perhaps create a script like his from scratch. Maybe a flag I missed for wget or something similar? Anyone?

10-03-2004, 04:28 AM	#2 (permalink)
greffov Da House Nerd Join Date: Dec 1969 Location: One CPU Lane Posts: 3,512	ah, that's simple. either telnet to the webserver, so you can see the headers (including gzip compression flag) or write some script to do so. you can do it simply with lynx Code: lynx -head http://www.pctechtalk.com/forums/ which will result in something like: Code: HTTP/1.1 200 OK Date: Sun, 03 Oct 2004 08:27:38 GMT Server: Apache/2.0.46 (Red Hat) Accept-Ranges: bytes X-Powered-By: PHP/4.3.2 Set-Cookie: ; path=/; domain=.pctec htalk.com Set-Cookie: bblastvisit=1096792058; expires=Mon, 03-Oct-2005 08:27:38 GMT; path =/; domain=.pctechtalk.com Content-Encoding: gzip Connection: close Content-Type: text/html __________________ Linux virusscanner detected a virus: Windows 95 ... delete [Y/n] y ~ ~ :wq

10-03-2004, 05:02 AM	#3 (permalink)
section31 Registered User Join Date: Sep 2004 Posts: 35	hm...for some reason out of all the forums ii use..yahoo always throws emails from this forum to the bulk mail...hence taking me so long to reply to this...sorry hm...thats just to find the headers, notice on that site, he also is able to find out how many bytes it is compressed/uncompressed. Last edited by section31; 10-03-2004 at 08:57 AM.

10-03-2004, 05:49 AM	#4 (permalink)
greffov Da House Nerd Join Date: Dec 1969 Location: One CPU Lane Posts: 3,512	yeah, well, that's simple. They retrieve the whole site, then gunzip it and calculate the ratio. If you really want I can make a PHP site that does the same as theirs. You can do it easily with Java as well. I only expect this functionality is only needed a few times so not worth the coding... __________________ Linux virusscanner detected a virus: Windows 95 ... delete [Y/n] y ~ ~ :wq

10-03-2004, 10:46 AM	#6 (permalink)
greffov Da House Nerd Join Date: Dec 1969 Location: One CPU Lane Posts: 3,512	You can never predict the compression rate of a zip algorithm. (g)Zip works by existance of duplicates of data, if none exist, the zip algorithm will not be able to be of any use. The level (1-9) affects the greedyness of the algorithm, which can result in a better compression ratio, but takes more CPU and memory. The only way to calculate a compression ratio is to compare the compressed and uncompressed data. __________________ Linux virusscanner detected a virus: Windows 95 ... delete [Y/n] y ~ ~ :wq

10-03-2004, 01:22 PM	#7 (permalink)
section31 Registered User Join Date: Sep 2004 Posts: 35	I see....so basically what you're saying is that, the site just gives an estimation of what it is compressed to. Since there is no way to predict what compression mode the server is using.

10-03-2004, 01:26 PM	#8 (permalink)
greffov Da House Nerd Join Date: Dec 1969 Location: One CPU Lane Posts: 3,512	I think the site is just retrieving the website. It's not that hard, and it enables calculation of the compression rate. The rates differ for me for several pages from the same website, so that makes me feel they really don't estimate anything, but just calculate the value. __________________ Linux virusscanner detected a virus: Windows 95 ... delete [Y/n] y ~ ~ :wq

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode