[Zlib-devel] DEFLATE performance improvements

[Jim Kukunas]

On Tue, Nov 26, 2013 at 11:59:55PM -0800, Mark Adler wrote:
> Jim,

Hi Mark,

> Thanks!  That's a lot to digest all at once.
> 
> Did Intel support this work?  How should I attribute it?

Yes. To Intel.

> For your speed comparisons, what compiler did you use on what kind of system?

The numbers were collected on a x86_64 Linux, 3.5.0, system with GCC
4.7.2.

> I don't really like the idea of an environment variable overriding the compression level, or worse, causing zlib to fail.  What is the rationale for that?

The rationale was that not all software that utilizes zlib exposes the
compression level as a tunable. This would allow the system
administrator to change the compression level without recompiling those
applications.

Thanks.

> On Nov 25, 2013, at 2:21 PM, Jim Kukunas <james.t.kukunas at linux.intel.com> wrote:
> > 
> > Hi Folks,
> > 
> > This patch series introduces a number of deflate performance improvements.
> > These improvements include two new deflate strategies, quick and medium,
> > as well as various improvements such as a faster hash function,
> > PCLMULQDQ-optimized CRC folding, and SSE2 hash shifting. 
> > 
> > The first patch adds a configure script check for determining the target
> > architecture. All optimizations added are encapsulated within preprocessor
> > checks which are selectively enabled based on target support. As such,
> > none of the patches in this series should have any effect on
> > non-x86{,_64} architectures.
> > 
> > The eleventh patch adds support for a new environmental variable ZLIB_LEVEL,
> > providing a simple method for system administrators to override the selected
> > compression level.
> > 
> > Obviously the performance is data dependent, however a few general trends:
> > 
> > Compression Corpora: Calgary, normal and large, Canterbury, normal and large,
> >                     and Silesia
> > Processor:  Intel i5-2530M @ 2.60 GHz with files stored in tmpfs to remove
> >            I/O overhead.
> > Compared with git commit 50893291621658f355bc5b4d450a8d06a563053d
> > 
> > Level 9 is about 22% faster with no change in compression.
> > 
> > Level 6 is about 50% faster with negligible change in compression.
> > 
> > Level 1 is about 71% faster with a sacrifice of about 30% compression.
> > 
> > Thanks.
> > 
> > --
> > Jim Kukunas
> > Intel Open Source Technology Center
> > 
> > 
> > 
> > _______________________________________________
> > Zlib-devel mailing list
> > Zlib-devel at madler.net
> > http://mail.madler.net/mailman/listinfo/zlib-devel_madler.net
> 
> 
> _______________________________________________
> Zlib-devel mailing list
> Zlib-devel at madler.net
> http://mail.madler.net/mailman/listinfo/zlib-devel_madler.net

-- 
Jim Kukunas
Intel Open Source Technology Center
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://madler.net/pipermail/zlib-devel_madler.net/attachments/20131127/358eee04/attachment.sig>