[Zlib-devel] Optimizations needed for gzgets() (Zlib version 1.2.3)

[Scott_Riley at amat.com]

Hi Mark,

Thanks again for your quick responses!

I think you hit the nail right on the head. The current application we have
integrated with the Zlib library requires processing files one line at a
time; therefore we need to use gzgets() for compressed files since we use
fgets() for uncompressed files. We cannot just read in blocks of data and
process them without a lot of manipulation and changes to the design. With
that said, is there a quicker way to decompress one line at a time? I have
not looked at inflateBack() yet because my understanding is that the first
requirement of not reading data line by line would have to be met first.

Please let me know your thoughts on this when you have a moment.

Regards,
Scott


The content of this message is Applied Materials Confidential.  If you are
not the intended recipient and have received this message in error, any use
or distribution is prohibited.  Please notify me immediately by reply
e-mail and delete this message from your computer system.  Thank you.


                                                                          
                                                                          
                                                                          
                                                                       To 
    Mark Adler                          zlib-devel at madler.net             
    <madler at alumni.caltech.edu>                                        cc 
    Sent by:                                                              
    zlib-devel-bounces at madler.n                                   Subject 
    et                                  Re: [Zlib-devel] Optimizations    
                                        needed for gzgets() (Zlib        
                                        version	1.2.3)                   
    01/06/2010 10:50 AM                                                   
                                                                          
                                                                          
         Please respond to                                                
       zlib-devel at madler.net                                              
                                                                          
                                                                          
                                                                          
                                                                          




On Jan 6, 2010, at 8:53 AM, Scott_Riley at amat.com wrote:
> Is there another library that will decompress these gzip compressed files
> faster than Zlib or do they need to uncompress the files and then use the
> compression and decompression routines from another library such as Lzop?

For lzop they would need to recompress the data.

There is no open source code faster than zlib to decompress gzip data that
I'm aware of.  (I haven't tested any commercial products, so it is possible
that something out there is faster.)  There are however options within zlib
to decrease the decompression time.  First off, unless you really need to
know where the end of each line is, you shouldn't use gzgets(), since that
just wastes time looking for the end-of-line characters.  Second, if you
are decompressing from a file, you can consider the use of inflateBack()
which uses a call-back interface to reduce some of the data-copying
overhead of inflate().  You would use inflate() before that to process the
gzip header, since inflateBack() only processes the raw deflate data.
There is also some assembler code to potentially speed up inflate in the
contrib directory, though I have not tested those.

Mark


_______________________________________________
Zlib-devel mailing list
Zlib-devel at madler.net
http://mail.madler.net/mailman/listinfo/zlib-devel_madler.net