[Zlib-devel] crc32 big/little endian

[Joakim Tjernlund]

>
> On Apr 21, 2010, at 7:48 AM, Joakim Tjernlund wrote:
> > Sure you can, use #ifdef __BYTE_ORDER == __BIG_ENDIAN like any
> > other app does.
>
> A perfect example of something that's not portable.  Not all platforms will
> define those symbols.  On my system, I can only find those in the X11 headers,
> and X11 is not installed by default on Mac OS X.

Surely there must be some other symbols then for OS X?

>
> > In what OS would ppc do that?
>
> Yes, there are applications that do that (the OS can't prevent it), and simply
> the fact that there exist processors that can do it makes code that depends on
> compile time decisions not portable.  zlib needs to be portable.

Oh, you are saying that an app changes the endian only for itself and the
rest of the OS and other apps are using Big Endian?
Either way, those few select apps could instead define NOBYFOUR

>
> > I don't think 16KB+ as small overhead, it is a lot.
>
> Then you can modify the code for your application.  That's the beauty of open

Actually I joined this list because I figured that zlib could benefit from some
of the linux kernel work I have done on crc32 and inflate, not because I need it for
some random app.

The beauty of open source is also that one can contribute improvements back that
others may benefit from.

You don't seem to see it that way, instead you seem to hide zlib development from the rest
of the world:
 - one cannot view the devel email archives or post unless one is a member
 - There is no repo were one can see the latest development.
 - There is no archive that I can find of zlib at gzip.org, posting there one
   gets a bounce message back from you.
So one begs to wonder if you really want help from outsiders like me.

> source.  By the way, the total table size is 16K, but you'd need to keep 8K
> for a particular endian choice so you'd only save 8K, plus a tiny amount of code space.
>
> (It's good we're having this discussion, because I just noticed that really
> the tables really only need to be four bytes per entry, not eight, so those
> numbers could all be cut in half.  On the other hand, it may be faster for the
> machine to access the native word size in the array when 64 bits.  On the
> other hand, the table would take less cache space if it were half the size.
> Any opinions on that?  Should we force the tables to use a four-byte integer type?)

Yes, cache misses costs a lot more so the size should not be bigger than it has too.

>
> > It would be better if there are any freak OS out there that can change endian on
> > the fly define NOBYFOUR instead of making all the rest suffer.
>
> On the vast majority of platforms with memory measured in gigabytes (a
> difference of six orders of magnitude!) there is no suffering.  Only in
> embedded applications with limited memory could that cause any suffering.
> Those applications can customize the code.  The presence of the unused portion
> of the table does not make the code slower, since only the part that's used
> occupies cache space.

Embedded devices vastly outnumbers standard computers
and these do not have gigs of memory. How many that uses zlib I have no clue
but I would bet that quite a few of them do but then again, embedded systems
doesn't seem to be something you are interested in.

>
> I am not clear on the suffering you refer to.

Every app that doesn't swap endian wastes memory.

  Jocke