[Zlib-devel] DEFLATE performance improvements

[Mark Adler]

Jim,

Thanks!  That's a lot to digest all at once.

Did Intel support this work?  How should I attribute it?

For your speed comparisons, what compiler did you use on what kind of system?

I don't really like the idea of an environment variable overriding the compression level, or worse, causing zlib to fail.  What is the rationale for that?

Mark


On Nov 25, 2013, at 2:21 PM, Jim Kukunas <james.t.kukunas at linux.intel.com> wrote:
> 
> Hi Folks,
> 
> This patch series introduces a number of deflate performance improvements.
> These improvements include two new deflate strategies, quick and medium,
> as well as various improvements such as a faster hash function,
> PCLMULQDQ-optimized CRC folding, and SSE2 hash shifting. 
> 
> The first patch adds a configure script check for determining the target
> architecture. All optimizations added are encapsulated within preprocessor
> checks which are selectively enabled based on target support. As such,
> none of the patches in this series should have any effect on
> non-x86{,_64} architectures.
> 
> The eleventh patch adds support for a new environmental variable ZLIB_LEVEL,
> providing a simple method for system administrators to override the selected
> compression level.
> 
> Obviously the performance is data dependent, however a few general trends:
> 
> Compression Corpora: Calgary, normal and large, Canterbury, normal and large,
>                     and Silesia
> Processor:  Intel i5-2530M @ 2.60 GHz with files stored in tmpfs to remove
>            I/O overhead.
> Compared with git commit 50893291621658f355bc5b4d450a8d06a563053d
> 
> Level 9 is about 22% faster with no change in compression.
> 
> Level 6 is about 50% faster with negligible change in compression.
> 
> Level 1 is about 71% faster with a sacrifice of about 30% compression.
> 
> Thanks.
> 
> --
> Jim Kukunas
> Intel Open Source Technology Center
> 
> 
> 
> _______________________________________________
> Zlib-devel mailing list
> Zlib-devel at madler.net
> http://mail.madler.net/mailman/listinfo/zlib-devel_madler.net