Re: XLF: Legacy Log Problems

John Cowan (cowan@locke.ccil.org)
Fri, 10 Jul 1998 09:21:11 -0400


Martin Gleeson wrote:

> This is the one concern I have with XLF. I already do the above with my
> web log analysis tool, pwebstats. And that's the problem - for example,
> the logs from our proxy server are 600Mb per week, gzipped, -9, and
> increasing. Whilst one of my next tasks is to have pwebstats handle
> incremental updates, it still shows the scope of the problem, especially
> for those who may need to keep a permanent copy of their logs for whatever
> reason. An order of magnitude increase in size creates difficulties when
> a feature of many logfile formats is representing the necessary information
> in the smallest reasonable space.

I understand your problem. However, even a 10-fold increase in
uncompressed size will *not* translate into a 10-fold increase in
compressed size, because most of the added material will be intensely
repetitive: tags, attribute names, markup characters.

-- 
John Cowan	http://www.ccil.org/~cowan		cowan@ccil.org
	You tollerday donsk?  N.  You tolkatiff scowegian?  Nn.
	You spigotty anglease?  Nnn.  You phonio saxo?  Nnnn.
		Clear all so!  'Tis a Jute.... (Finnegans Wake 16.5)