[Zlib-devel] Compressing volatile data?

[Mark Adler]

On Sep 3, 2010, at 6:12 AM, Gary Cameron wrote:
> It was obvious the checksum could be wrong in this case, but I wasn't sure if it could explain decompression errors as well.

Gary,

No, this problem would not be able to explain invalid deflate data such as the inflate messages (returned in strm->msg) "invalid block type", "invalid stored block lengths", "too many length or distance symbols", "invalid code lengths set", "invalid bit length repeat", "invalid code -- missing end-of-block", "invalid literal/lengths set", "invalid distances set", "invalid literal/length code", "invalid distance code", or "invalid distance too far back".

It does however explain the message "incorrect data check".

All of those have the same return code from inflate (Z_DATA_ERROR).

On Sep 3, 2010, at 7:53 AM, glennrp at comcast.net wrote:
>         Don't you still hit the zmemcpy twice, 
>         recreating the buf for each 'wrap'?

No, the zmemcpy() is called only once for any given input data.  Only one of the check value calls is made, depending on whether a zlib stream or gzip stream is being generated.  Though even if both were called, with the fix they would both be accessing the non-volatile internal zlib buffer.

On Sep 3, 2010, at 8:08 AM, Gary Cameron wrote:
> Your proposed zmemcpy fix is a great safety feature that should probably be enabled by default, although he was concerned if there would be an unnecessary performance hit from the extra copy if you could guarantee the source data was static.

There is no performance hit, since the zmemcpy() has always been there.  The only change is moving the check computation after the zmemcpy() so that it can use the internal buffer.

That fix will be the default and only behavior in the next version of zlib.  Thank you for reporting this.  Indeed, the fix makes deflate more robust.

Mark