[Zlib-devel] patch-in-progress: vectorized adler32 calculation

[Stefan Fuhrmann]

Greg Roelofs wrote:
> Stefan Fuhrmann <stefanfuhrmann at alice-dsl.de> wrote:
>> he overall performance of the deflate() function is already quite
>> impressive: For real-world repository data,
>>     
Dang! I meant *inflate*. Extracting data from the repository is my
main concern.
> Primarily C/C++ source code?
>   
No. Next to the source code, there is more or less incompressible
binary data (e.g. PNG files for documentation).
>> I measured ~200MB inflated data per sec.
>>     
>
> Does that include I/O overhead or is it for data already in memory and/or
> L1/L2 caches?  And you really are referring to the rate of _output_ data,
> right?  So it's on the order of 600MB/sec for input-data processing speed
> (assuming the typical 3x or so compression factor for source code)?
>   
The data is at least in memory already and probably in L3/L2
(just got read from the OS file cache). My measurement was
that 120MB of decompressed file delta info gets produced in
.6 seconds. Since these deltas get combined to 85MB of final
file content, there is little room for reducing the data that inflate
has to process.
> Cool results.
>   
Or maybe not. They imply that no major performance gains
can be expected here.

-- Stefan^2.