[Zlib-devel] HEADS UP: Apparent bad compilation under (just released) GCC 4.5.0

[Török Edwin]

On 05/11/2010 10:13 PM, Jesus Cea wrote:
> On 04/05/10 20:20, Török Edwin wrote:
>> Which optimization level breaks it? Does it work with -O2? -O1?
>> You can try to find out which individual optimization flag breaks it, by
>> decomposing -O2 into individual optimization flags, and using a binary
>> search to find out which flag is the culprit.
> 
> Sorry, I forgot to comment that. It fails with -O3, but work with -O2.
> 
> Binary search would only garantee a resolution if optimizations where
> independient. But it is a good try anyway. Let's see...
> 
>> If this happens at -O3 only, my usual suspect is -ftree-vectorize.
> 
> Uhmmm...

Yeah it tends to expose bugs elsewhere (not in the vectorization itself,
but in assumptions about alignment).

> 
>> For example you can decompose -O3 into individual flags by running this
>> command:
>> gcc-4.5 -O3 -Q --help=optimizers|grep enabled|cut -f1 -d\[|xargs echo
> 
> I was not aware of how to know what optimizations every level (-O)
> enables. That was a very useful tip. Thanks.
> 
>> Does the bug occur if you try to compress/uncompress the file using
>> minigzip?
>> Or can you find another way to reproduce the crash with python only
>> (without psyco)?
> 
> No, I have been not able to reproduce the crash in other setups. I guess
> psyco JIT code generation is leaving some register in a state GCC
> doesn't like. Or the other way around.
> 
>> Can you take a stacktrace, or disassemble the code around the crash (if
>> the stacktrace doesn't show useful line numbers)?
> 
> The crash corrupts memory, so the place where it fails is not the actual
> wrong code. Nontrivial to debug without special tools like Valgrind or a
> CPU simulator.
> 
>> I was going to suggest running under valgrind, but I doubt valgrind
>> supports Solaris.
> 
> I thought the same. No, Valgrind doesn't support Solaris. But since this
> machine is x86, I guess the generated could fail the same way... But I
> don't have any Linux/x86 machine with GCC 4.5.0. Not easy upgrade path
> either. Too risky.
> 
> I was thinking about installing a linux virtual machine and installing
> there GCC 4.5.0, but I have had a pretty busy week.
> 
> OK, the situation is this:
> 
> * Library works with -O2.
> 
> * Library doesn't work with -O3.
> 
> * Library doesn't work with -O3 -fno-strict-aliasing.
> 
> * Library works with -O3 -fno-tree-vectorize.
> 
> * Library doesn't work with -O2 -ftree-vectorize.
> 
> * Library doesn't work with -O1 -ftree-vectorize.
> 
> * Library works with -O0 -ftree-vectorize.
> 
> So, it is pretty clear that "-ftree-vectorize" is an issue.
> 
> Comparing with GCC 4.4.3 in other machine, I see "-ftree-vectorize"
> activated there too. So the feature is actually "old", but seems to have
> some regression in 4.5.0.
> 
> Uhmmmm... listing ASM, difference between 4.4.3 and 4.5.0 is pretty big,
> so "diff" is not useful to me. And I am not able to debug 86 code myself
> :-(.
> 
> The bad compilation seems to be in "inftree.c", but it is too big (442
> instructions) to decipher by hand (by me).
> 
> Seems to be some noise about this online:
> 
> * Reference to zlib 1.2.5:
> http://forums.gentoo.org/viewtopic-p-6262832.html?sid=13c94bd097c3785d43a4275158bbbaba#6262832
> 
> * Reference to bugs in "-ftree-vectorize":
> http://www.phoronix.com/forums/showthread.php?t=23239&page=4
> 
> * The same synthoms: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42513 ,
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40838 .
> 
> So, my problem seems to be a know flaw in 4.5.0.
> 
> There is a suggestion to add "-mstackrealign". It seems to work now with
> stock "-O3 -stackrealign".
> 
> So, this is a know issue. I will stop here.
> 
> PS: For future reference, I reproduce this with:
> 
> """
> $ export LD_LIBRARY_PATH=/tmp/zlib-1.2.5  # ZLIB under test
> $ python
>>>> import psyco
>>>> psyco.full()
>>>> import pytz  # Or any other library living in a .zip file
> (core dump)

OK, maybe psyco doesn't align the stack as GCC expects it?
Please discuss this with psyco developers.

It doesn't look like zlib itself is doing anything wrong here.

Best regards,
--Edwin