[Zlib-devel] Optimizations needed for gzgets() (Zlib version 1.2.3)

[Scott_Riley at amat.com]

Thank you Gilles,

I was actually looking at the code late yesterday and noticed the same
about gzgets(). The code you provided below will give me a good head start
on optimizing this feature in my application.

I greatly appreciate your feedback.

Regards,
Scott


The content of this message is Applied Materials Confidential.  If you are
not the intended recipient and have received this message in error, any use
or distribution is prohibited.  Please notify me immediately by reply
e-mail and delete this message from your computer system.  Thank you.


                                                                          
                                                                          
                                                                          
                                                                       To 
    "Gilles Vollant"                    <zlib-devel at madler.net>           
    <info at winimage.com>                                                cc 
    Sent by:                                                              
    zlib-devel-bounces at madler.n                                   Subject 
    et                                  Re: [Zlib-devel] Optimizations    
                                        needed for gzgets()	(Zlib        
                                        version	1.2.3)                   
    01/07/2010 05:38 AM                                                   
                                                                          
                                                                          
         Please respond to                                                
       zlib-devel at madler.net                                              
                                                                          
                                                                          
                                                                          
                                                                          




gzgets call gzread one time for each char.

I suggest you use instead gzread with a bigger buffer (0x100 bytes or more,
by example)


just to give an idea, here is a buffered gets I wrote for another
application.
there is a af_fseek you must remove (not fast fseek on gzio.c), instead use
a persistent tab_in and read_ascii_in_file



#define BUFFER_IN_CACHE_SIZE (0x100)

int u_fgets_buffered(char* line,int i_is_size,int size,FILE* f)
{
  unsigned char tab_in[BUFFER_IN_CACHE_SIZE];
  size_t read_ascii_in_file = 0;
  int pos_in_char_line = 0;

  if ((i_is_size != 0) && (size == 0))
    return EOF;



  for (;;)
  {
		   int size_to_read_binary = BUFFER_IN_CACHE_SIZE;
		   if (i_is_size != 0)
		 		   if ((size-1) < BUFFER_IN_CACHE_SIZE)
		 		 		   size_to_read_binary = size-1;

		   if (size_to_read_binary>0)
		 		   read_ascii_in_file =
fread(&tab_in[0],1,(size_t)size_to_read_binary,f);
		   if (read_ascii_in_file<=0)
		   {
		 		   if (pos_in_char_line == 0)
		 		 		   return EOF;
		 		   else
		 		   {
		 		 		   line[pos_in_char_line]='\0';
		 		 		   return pos_in_char_line;
		 		   }
		   }

		   if (read_ascii_in_file > 0)
		   {
		 		   int i;
		 		   for (i=0;i<(int)read_ascii_in_file;i++)
		 		   {
		 		 		   char c;
		 		 		   c = (((char)tab_in[(i)])) ;

		 		 		   if (c!=0x0d)
		 		 		   {
		 		 		 		   if (c=='\n')
		 		 		 		   {
		 		 		 		 		   fseek(f,-1 *
(long)(read_ascii_in_file - (i+1)),SEEK_CUR);
		 		 		 		 		   if
(i_is_size!=0) {

/* only if you want \n at
end of string : line[pos_in_char_line++]='\n';*/
		 		 		 		 		   }
		 		 		 		 		   line
[pos_in_char_line]='\0';
		 		 		 		 		   return
pos_in_char_line;
		 		 		 		   }

		 		 		 		   if ((i_is_size==1) &&
(pos_in_char_line ==
(size-1)))
		 		 		 		   {
		 		 		 		 		   fseek(f,-1 *
(long)(read_ascii_in_file - i),SEEK_CUR);
		 		 		 		 		   line
[pos_in_char_line]='\0';
		 		 		 		 		   return
pos_in_char_line;
		 		 		 		   }

		 		 		 		   line[pos_in_char_line++]
= c;
		 		 		   }
		 		   }
		   }
		   read_ascii_in_file = 0;

  }
}

-----Message d'origine-----
De : zlib-devel-bounces at madler.net [mailto:zlib-devel-bounces at madler.net]
De
la part de Enrico Weigelt
Envoyé : jeudi 7 janvier 2010 01:32
À : zlib-devel at madler.net
Objet : Re: [Zlib-devel] Optimizations needed for gzgets() (Zlib version
1.2.3)

Scott_Riley at amat.com wrote:

> I think you hit the nail right on the head. The current application we
have
> integrated with the Zlib library requires processing files one line at a
> time; therefore we need to use gzgets() for compressed files since we use
> fgets() for uncompressed files.

hmm, maybe you could call zcat as subprocess and assign its output
to a FILE ? ;-o

BTW: if you might consider rewriting the app to some vfs layer, you
could use libmvfs - it doesnt have zlib support right now, but I'm
going to add it soon anyways.





_______________________________________________
Zlib-devel mailing list
Zlib-devel at madler.net
http://mail.madler.net/mailman/listinfo/zlib-devel_madler.net