[Zlib-devel] zlib large file support

[Nelson H. F. Beebe]

Mark Adler <madler at alumni.caltech.edu> asks about compilation of zlib
on various systems to enable access to 64-bit file offsets.

There are two approaches: build the library in a 32-bit world with
64-bit file offsets, or build the library in a 64-bit world.

On some systems, both libraries may be needed: GNU/Linux on AMD64, SGI
IRIX, and Sun Solaris all support both 32-bit and 64-bit worlds.  The
default world on AMD64 is 64-bit, and on SGI and Sun, it is 32-bit.

For a 64-bit world:

==============================================================================================================
Vendor and O/S		Compiler(s)		Options				Library installation directory
==============================================================================================================
Alpha GNU/Linux		gcc			(default)			$(prefix)/lib
AMD64 GNU/Linux		gcc			(default)			$(prefix)/lib64
						-m32				$(prefix)/lib
IA-32 GNU/Linux		gcc			n/a				n/a
IA-64 GNU/Linux		gcc			(default)			$(prefix)/lib
MIPS GNU/Linux		gcc			n/a				n/a
PowerPC GNU/Linux	gcc			n/a				n/a
SPARC GNU/Linux		gcc			n/a				n/a
PowerPC Apple MacOS 10	gcc			n/a				n/a
IA-32 FreeBSD 4, 5	gcc			n/a				n/a
IA-32 OpenBSD 3.2	gcc			n/a				n/a
IA-32 NetBSD 1.6	gcc			n/a				n/a
Compaq/DEC/HP Alpha	gcc, cc			(default)			$(prefix)/lib
   OSF/1 4, 5
IA-64 HP/UX 11.x	gcc, cc			(default)			$(prefix)/lib
PA/RISC HP/UX 10.x,11.x gcc, cc			+DD64				$(prefix)/lib/pa20_64
IBM AIX	4.x		cc			(default)			$(prefix)/lib
SGI IRIX 6.x		c89, cc and gcc		-64				$(prefix)/lib64
Sun Solaris 7, 8, 9	cc, c89			-xtarget=ultra -xarch=v9	$(prefix)/lib64
Sun Solaris 7, 8, 9	gcc			-m64				$(prefix)/lib64
Sun Solaris 10		cc, c89, c99		-xtarget=ultra -xarch=v9	$(prefix)/lib/64
Sun Solaris 10		gcc			-m64				$(prefix)/lib/64
==============================================================================================================

Notice the changed library directory in Solaris 10 compared to earlier
releases: lib/64 instead of lib64.

The GNU/Linux systems on MIPS, PowerPC, and SPARC fail to provide
access to a 64-bit world, even though the hardware supports it.

The AMD64 in forced 32-bit mode (with cc -m32) is an IA-32 system,
with system libraries in /usr/lib.  In its default 64-bit mode (or
with cc -m64), it is an AMD64 system with extended instruction set,
sixteen 128-bit directly-addressable floating-point registers that do
32-bit and 64-bit arithmetic in bundles of one, two, or four at a time
in parallel.  The C/C++ long double type on AMD64 is an 80-bit one, as
on IA-32 and IA-64, and the IA-32 floating-point instruction set with
a stack of eight floating-point registers must be used for arithmetic.

For a 32-bit world, it may be necessary to use a different data type,
off64_t, to get large file support, and on GNU/Linux, the compiler
option -D_LARGEFILE64_SOURCE is required on ALL architectures (even
64-bit ones) to make that datatype visible in <sys/types.h>.

=================================================================================
Vendor/Architecture and O/S		sizeof(off_t)	sizeof(off64_t)
=================================================================================
Alpha GNU/Linux				8		8 (-D_LARGEFILE64_SOURCE)
AMD64 GNU/Linux				8 (default)	8 (-D_LARGEFILE64_SOURCE)
AMD64 GNU/Linux				4 (-m32)	8 (-D_LARGEFILE64_SOURCE)
IA-32 GNU/Linux				4		8 (-D_LARGEFILE64_SOURCE)
IA-64 GNU/Linux				8		8 (-D_LARGEFILE64_SOURCE)
MIPS GNU/Linux				4		8 (-D_LARGEFILE64_SOURCE)
PowerPC GNU/Linux			4		8 (-D_LARGEFILE64_SOURCE)
SPARC GNU/Linux				4		8 (-D_LARGEFILE64_SOURCE)
PowerPC Apple MacOS 10			8		n/a
IA-32 FreeBSD 4, 5			8		n/a
IA-32 OpenBSD 3.2			8		n/a
IA-32 NetBSD 1.6			8		n/a
Compaq/DEC/HP Alpha OSF/1 4, 5	 	8		n/a
IA-64 HP/UX 11.x			4		8
PA/RISC HP/UX 10.x, 11.x		4		8
IBM AIX	4.x				4		8
SGI IRIX 6.x				8		8
Sun Solaris 7, 8, 9, 10 SPARC		4		8
Sun Solaris 9 x86			4		8
=================================================================================

For reference, here is the little test program that I ran today on all
of the above systems:

% cat off_t.c
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>

int
main(void)
{
    printf("sizeof(off_t)   = %u\n", sizeof(off_t));

#if defined(M64)
    printf("sizeof(off64_t) = %u\n", sizeof(off64_t));
#endif

    return (EXIT_SUCCESS);
}

Compilations went like this (e.g., on Sun Solaris 8):

	% cc -DM64 off_t.c && ./a.out
	sizeof(off_t)   = 4
	sizeof(off64_t) = 8

	% cc off_t.c && ./a.out
	sizeof(off_t)   = 4

Plug: For much more detailed probing of your own C and C++ compilation
environments, try the features package at

	http://www.math.utah.edu/pub/features/

-------------------------------------------------------------------------------
- Nelson H. F. Beebe                    Tel: +1 801 581 5254                  -
- University of Utah                    FAX: +1 801 581 4148                  -
- Department of Mathematics, 110 LCB    Internet e-mail: beebe at math.utah.edu  -
- 155 S 1400 E RM 233                       beebe at acm.org  beebe at computer.org -
- Salt Lake City, UT 84112-0090, USA    URL: http://www.math.utah.edu/~beebe  -
-------------------------------------------------------------------------------