Difference between GCC and Clang

[Richard Russell]

I'm seeing this, superficially surprising, difference in behaviour between different editions of BBC BASIC for SDL 2.0 when presented with the statement a%% = 2^63:

GCC targetting x86 (32-bit or 64-bit): 'Number too big'
GCC targetting ARM (32-bit or 64-bit): 'Number too big'
Clang targetting x86 (64-bit): 'Number too big'
Clang targetting ARM (64-bit): No error (a%% set to &7FFFFFFFFFFFFFFF)!
Clang targetting web assembly: 'Number too big'

The code I'm using for the test is this:

	long long t = v.f ;
	if (t != truncl(v.f))
		error (20, NULL) ; // 'Number too big'

Presumably some Undefined Behaviour is resulting in the difference with Clang targetting 64-bit ARM. Can anybody suggest an alternative method which will work reliably in all cases?

Soruk

I don't have an ARM64 system/setup to play with, but a little thought...
Can you make it print off the values (either suitable printf()s or a gdb breakpoint) of v.f, t and truncl(v.f)?

(Aside: I get different results in Matrix Brandy for 2^63, on x86 and x86_64 I get "Number is out of range", on ARM32 I get &8000000000000000 - which, in a sense, isn't wrong. Edit - this was with GCC.)

[Richard Russell]

Soruk wrote: »

Can you make it print off the values (either suitable printf()s or a gdb breakpoint) of v.f, t and truncl(v.f)?

Since it only 'fails' here in iOS this isn't at all easy. For a start I don't know how I would monitor the output of printf in iOS (in Android I'd use logcat but I don't have a clue about iOS debugging).

(Aside: I get different results in Matrix Brandy for 2^63, on x86 and x86_64 I get "Number is out of range", on ARM32 I get &8000000000000000 - which, in a sense, isn't wrong.)

Is that using GCC in both cases? I've not (yet) seen any inconsistencies like that with GCC, only with Clang (and it surely is wrong because that hex value is −2^63, not 2^63!).

Soruk

At some point I'll get an RPi4 and try a 64-bit OS on it. I'm currently running an RPi 3B+ running a 32-bit ARM OS. And yes, I was using GCC in all cases. (And, I get the same thing on ARM32 and ARM26 under RISC OS.)

Could your issue be that ARM doesn't support 80-bit floats in the hardware? That's just a straw being clutched! As you can't printf or logcat, you could perhaps print to the BBC display?

[Richard Russell]

Soruk wrote: »

Could your issue be that ARM doesn't support 80-bit floats in the hardware?

I don't see how that's connected, since it's equally true for GCC and Clang and the 'fault' only occurs in Clang! One reason why I chose the value 2^63 for the demonstration is because it can be precisely represented both as a double (64-bits) and as a long double (80-bits).

Anyway I said in my original post what the "issue" is: it has to be reliance on Undefined Behaviour (always assuming it's not a compiler or library bug).

My question was (and is): how can I modify the test to make it legitimate, in the sense of not relying on UB and therefore, hopefully, working equally well on every compiler and every platform?

[Richard Russell]

Soruk wrote: »

At some point I'll get an RPi4 and try a 64-bit OS on it.

Incidentally be careful if you do any tests on a 64-bit RPi4 because long double on that platform is neither 64-bits (as it is on 32-bit ARM) nor 80-bits (as it is on x86) but 128-bits! That's why I explicitly set my float type rather than relying on the default for the platform:

#if defined(__arm__) || defined(__aarch64__) || defined(__EMSCRIPTEN__)
	double f ;
#else
        long double f ;
#endif

Soruk

Back to your original point, having realised I can run Raspbian 64-bit on a RasPi 3B (so no need to buy a RPi4)....

(mclang is a copy of the makefile, altered to use clang instead of gcc.)

soruk@raspberrypi:~/git/BBCSDL/console/rpi64 $ uname -a
Linux raspberrypi 5.15.32-v8+ #1538 SMP PREEMPT Thu Mar 31 19:40:39 BST 2022 aarch64 GNU/Linux
soruk@raspberrypi:~/git/BBCSDL/console/rpi64 $ make -f mclang
clang -fPIC -Wall -I ../../include -I ../../../BBCSDL/include -Wno-attributes -c -O2 ../../src/bbmain.c -o bbmain.o
clang -fPIC -Wall -I ../../include -I ../../../BBCSDL/include -c -O2 ../../src/bbexec.c -o bbexec.o
clang -fPIC -Wall -I ../../include -I ../../../BBCSDL/include -Wno-array-bounds -c -O2 ../../src/bbeval.c -o bbeval.o
clang -fPIC -Wall -I ../../include -I ../../../BBCSDL/include -c -Os ../../src/bbasmb_arm_64.c -o bbasmb.o
sed 's/_\([a-z]*\)/\1/g' <../../../BBCSDL/src/bbdata_arm_64.s >bbdata.s
as bbdata.s -o bbdata.o
clang -fPIC -Wall -I ../../include -I ../../../BBCSDL/include -Wno-array-bounds -Wno-unused-result -c -Os ../../src/bbccos.c -o bbccos.o
clang -fPIC -Wall -I ../../include -I ../../../BBCSDL/include -Wno-array-bounds -Wno-unused-result -c -O2 ../../src/bbccon.c -o bbccon.o
clang -fPIC -Wall -I ../../include -I ../../../BBCSDL/include -c -O3 ../../src/sort.c -o sort.o
clang -fPIC -Wall -I ../../include -I ../../../BBCSDL/include bbmain.o bbexec.o bbeval.o bbasmb.o bbdata.o bbccos.o bbccon.o sort.o -L . -L/usr/lib/ -ldl -lm -lrt -pthread \
-o bbcbasic -Wl,-s -Wl,-R,'$ORIGIN'
cp bbcbasic ../../
soruk@raspberrypi:~/git/BBCSDL/console/rpi64 $ ./bbcbasic 
BBC BASIC for Linux Console v0.41
(C) Copyright R. T. Russell, 2022
>a%%=2^63

Number too big
>_

[Richard Russell]

Yes, it's iOS in which it doesn't work:

Soruk

Yeah, it's curious, my thought at this point is it might be an iOS system library (or SDK component) issue then, since I don't think it's strictly a Clang on ARM64 issue, as the issue doesn't show up on Clang on ARM64 Linux.

[Richard Russell]

Soruk wrote: »

I don't think it's strictly a Clang on ARM64 issue, as the issue doesn't show up on Clang on ARM64 Linux.

I expect it's neither a Clang nor an ARM64 issue as such, but rather an Apple issue! For example as I said earlier long double is 128-bits in ARM64 Linux, but not ARM64 iOS in which it's 64-bits!

But the cause isn't really important, the point is that it doesn't work. If we neglect the remote possibility that it's a genuine compiler or library bug, the likelihood is that I'm relying on UB for it to work at all. Even on the platforms (the majority) where it does currently work, it might stop working without warning.

Soruk

It's why I was wondering if there was a way to display the intermediate values in your function, including the truncl() function. Perhaps (and I admit it's a very dirty hack) make a build for your phone/ipad that calls your print functions to display these. It might give a pointer (pun not intended) to where it may be going wrong.

[Richard Russell]

Soruk wrote: »

make a build for your phone/ipad that calls your print functions to display these.

I have no idea how I would do that. And even if I succeeded there are practical issues such as having (if only temporarily) to replace the version available online, for users to download, with the patched version (so somebody might download it by mistake).

This is because I don't know of any way of building a version with a different filename etc. without having to go through all the hoops of creating new certificates, device lists, provisioning profiles etc. (i.e. treating it like a completely new app).

It might give a pointer (pun not intended) to where it may be going wrong.

If, as I suspect, the cause is UB then nothing is "going wrong". In that case all that work would not tell us anything, since by definition Undefined Behaviour can do anything at all!

You seem to be convinced that it's not UB but a bug of some kind in the compiler, OS, libraries etc. What leads you to that conclusion? Are you of the opinion that the code I'm using is entirely legitimate and cannot trigger UB in the circumstances in which I am using it?

Soruk

I don't think I can really say either way, without knowing what's in t, v.f and the result of truncl(v.f). Does the Apple developer kit not provide an emulator to test code before uploading it to real hardware? I'm pretty sure the Android SDK has this.

Incidentally, with my unexpected sign-flip on ARM26 and ARM32, I added a check to see if the sign flipped, and if so, trigger the "Number is out of range" error. This way I'm now getting consistent behaviour on Intel and ARM - including ARM64.

[Richard Russell]

Soruk wrote: »

Does the Apple developer kit not provide an emulator to test code before uploading it to real hardware?

Probably. I've never needed to (or wanted to) find out. But if it's anything like the Android simulator it's not much use for debugging this kind of thing, because it's a functional simulator only. It doesn't run the same binary code, or even emulate the CPU!

For example if I run BBC BASIC for SDL 2.0 on the Android simulator I get an x86 CPU with 80-bit floats (and __ARM__ isn't defined). So it's a completely different environment, at a low level, from what will be running on a real device - and of course assembly-language code doesn't work.

I don't think I can really say either way, without knowing what's in t, v.f and the result of truncl(v.f).

I'm confused (not for the first time). We know from the code that t is equal to truncl(v.f) (at least, after a cast); if they were not equal the 'Number too big' error would be generated, and it isn't. Further, we know that v.f is 2^63, because that's the value the statement is assigning. Finally we know that t is &7FFFFFFFFFFFFFFF because that's the value which gets assigned to a%%.

So, surely, we already know all the things you are asking about. v.f is 2^63, t is &7FFFFFFFFFFFFFFF, and so is truncl(v.f). What am I missing?

[Richard Russell]

Richard_Russell wrote: »

We know from the code that t is equal to truncl(v.f) (at least, after a cast)

I've tried to find out how C compares a 64-bit double with a 64-bit integer. Seemingly it first converts the integer to a double, and then compares the two doubles. Of course this involves a loss of precision, because the double has only a 52-bit mantissa.

So it's entirely likely that the integer value &7FFFFFFFFFFFFFFF (2^63-1) will be converted to (exactly) 2^63 when promoted to a double, and the comparison will succeed.

This is what brings me back to the UB question. The correct functioning of my code (or not) crucially depends on how a float value of precisely 2^63 will be cast to a 64-bit integer (which cannot contain that value). I bet that's Undefined Behaviour in C.

(Distillery has become virtually unusable for me, again, because of the constant 'not responding' messages).

[Richard Russell]

I've written more at the Raspberry Pi forum here.

Soruk

Aside: I've been tweaking the TCP stack settings on the server, some might improve the situation, some might not... I'm testing (and posting) from a remote Linux machine, and my phone on a 4G connection, though again I am unable to reproduce any timeouts.