[Zlib-devel] crc32 big/little endian

[Mark Adler]

On Apr 21, 2010, at 7:48 AM, Joakim Tjernlund wrote:
> Sure you can, use #ifdef __BYTE_ORDER == __BIG_ENDIAN like any
> other app does.

A perfect example of something that's not portable.  Not all platforms will define those symbols.  On my system, I can only find those in the X11 headers, and X11 is not installed by default on Mac OS X.

> In what OS would ppc do that?

Yes, there are applications that do that (the OS can't prevent it), and simply the fact that there exist processors that can do it makes code that depends on compile time decisions not portable.  zlib needs to be portable.

> I don't think 16KB+ as small overhead, it is a lot.

Then you can modify the code for your application.  That's the beauty of open source.  By the way, the total table size is 16K, but you'd need to keep 8K for a particular endian choice so you'd only save 8K, plus a tiny amount of code space.

(It's good we're having this discussion, because I just noticed that really the tables really only need to be four bytes per entry, not eight, so those numbers could all be cut in half.  On the other hand, it may be faster for the machine to access the native word size in the array when 64 bits.  On the other hand, the table would take less cache space if it were half the size.  Any opinions on that?  Should we force the tables to use a four-byte integer type?)

> It would be better if there are any freak OS out there that can change endian on
> the fly define NOBYFOUR instead of making all the rest suffer.

On the vast majority of platforms with memory measured in gigabytes (a difference of six orders of magnitude!) there is no suffering.  Only in embedded applications with limited memory could that cause any suffering.  Those applications can customize the code.  The presence of the unused portion of the table does not make the code slower, since only the part that's used occupies cache space.

I am not clear on the suffering you refer to.

Mark