[Zlib-devel] Optimizations needed for gzgets() (Zlib version 1.2.3)

[Gilles Vollant]

gzgets call gzread one time for each char.

I suggest you use instead gzread with a bigger buffer (0x100 bytes or more,
by example)


just to give an idea, here is a buffered gets I wrote for another
application.
there is a af_fseek you must remove (not fast fseek on gzio.c), instead use
a persistent tab_in and read_ascii_in_file



#define BUFFER_IN_CACHE_SIZE (0x100)

int u_fgets_buffered(char* line,int i_is_size,int size,FILE* f)
{
  unsigned char tab_in[BUFFER_IN_CACHE_SIZE];
  size_t read_ascii_in_file = 0;
  int pos_in_char_line = 0;

  if ((i_is_size != 0) && (size == 0))
    return EOF;



  for (;;)
  {
	  int size_to_read_binary = BUFFER_IN_CACHE_SIZE;
	  if (i_is_size != 0)
		  if ((size-1) < BUFFER_IN_CACHE_SIZE)
			  size_to_read_binary = size-1;

	  if (size_to_read_binary>0)
		  read_ascii_in_file =
fread(&tab_in[0],1,(size_t)size_to_read_binary,f);
	  if (read_ascii_in_file<=0)
	  {
		  if (pos_in_char_line == 0)
			  return EOF;
		  else
		  {
			  line[pos_in_char_line]='\0';
			  return pos_in_char_line;
		  }
	  }

	  if (read_ascii_in_file > 0)
	  {
		  int i;
		  for (i=0;i<(int)read_ascii_in_file;i++)
		  {
			  char c;
			  c = (((char)tab_in[(i)])) ;

			  if (c!=0x0d)
			  {
				  if (c=='\n')
				  {
					  fseek(f,-1 *
(long)(read_ascii_in_file - (i+1)),SEEK_CUR);
					  if (i_is_size!=0) {
						  /* only if you want \n at
end of string : line[pos_in_char_line++]='\n';*/
					  }
					  line[pos_in_char_line]='\0';
					  return pos_in_char_line;
				  }

				  if ((i_is_size==1) && (pos_in_char_line ==
(size-1)))
				  {
					  fseek(f,-1 *
(long)(read_ascii_in_file - i),SEEK_CUR);
					  line[pos_in_char_line]='\0';
					  return pos_in_char_line;
				  }

				  line[pos_in_char_line++] = c;
			  }
		  }
	  }
	  read_ascii_in_file = 0;

  }
}

-----Message d'origine-----
De : zlib-devel-bounces at madler.net [mailto:zlib-devel-bounces at madler.net] De
la part de Enrico Weigelt
Envoyé : jeudi 7 janvier 2010 01:32
À : zlib-devel at madler.net
Objet : Re: [Zlib-devel] Optimizations needed for gzgets() (Zlib version
1.2.3)

Scott_Riley at amat.com wrote:

> I think you hit the nail right on the head. The current application we
have
> integrated with the Zlib library requires processing files one line at a
> time; therefore we need to use gzgets() for compressed files since we use
> fgets() for uncompressed files. 

hmm, maybe you could call zcat as subprocess and assign its output
to a FILE ? ;-o

BTW: if you might consider rewriting the app to some vfs layer, you
could use libmvfs - it doesnt have zlib support right now, but I'm
going to add it soon anyways.