[Zlib-devel] pigz 2.1.4 output differs for -p 1 and -p 2

[Mark Adler]

On Nov 13, 2008, at 2:40 PM, wpilorz at gmail.com wrote:
> My question is whether this is expected behaviour that pigz produces
> different compressed output when -p 1 and -p n with n > 1 is used,  
> even with -i?

Wojtek,

No, my intent is that the output be the same.

I have not seen any difference in the output when -i is *not* used.   
Are you seeing a difference?

I am however seeing a difference in the output when -i is used.  The  
difference in the code between -p 1 and -p 2 is that the former uses  
Z_FULL_FLUSH to end a block and start another, whereas the latter uses  
deflateReset() between blocks.  I did an experiment and added a  
deflateReset() in the -p 1 case, and now the output of -p 1 and -p 2  
are identical.

This behavior is odd.  In theory, Z_FULL_FLUSH should erase any memory  
of the previous data, yet somehow the size of the next block is  
different when using Z_FULL_FLUSH as opposed to deflateReset().   
deflateReset() is doing a "better" job of erasing the memory of the  
previous compression.  This may in fact be a bug in deflate in zlib.   
I will look into it.

Mark