Surprising indirection behaviour
![[Richard Russell]](https://distillery.matrixnetwork.co.uk:3004/applications/dashboard/design/images/defaulticon.png)
Over at the 'BASIC Programming Language' Facebook group there is a discussion about how BBC BASIC handles a!b!c. Paul Fellows seems to think that this is interpreted differently depending on whether it's encountered in a 'left' or 'right' context.
To test it I wrote this little program:
To test it I wrote this little program:
10 DIM a 100, b 100 20 c = 32 30 b!c = 50 40 50 a!b!c = 123456 60 PRINT a!b!c 70 a!b!c = 654321 80 PRINT a!b!cThat doesn't support his hypothesis; in every version of BBC BASIC I've tried (6502 BASIC 2, ARM BASIC 5, BB4W, BBCSDL - both assembler and C editions) it outputs this, which is consistent with what I would expect:
123456
654321
>
But in Matrix Brandy (64-bit) it reports Number is out of range at line 30! Is that expected?0
Comments
-
Ooh. A puzzle. I like these. Looks like I'm going to be code diving for the next few days!0
-
The plot thickens....
0 -
I can't replicate this on a Windows 64-bit build either. What is the git tag in your title bar?
0 -
Here's what I get (Windows 11, if it matters)
0 -
Actually I wouldn't be surprised if Windows 11 is a factor because I noticed with my BASICs that the heap was typically being allocated at a much higher address than with Windows 10. Now it's always above the range that can be addressed using a 32-bit pointer:Richard_Russell wrote: »Here's what I get (Windows 11, if it matters)
0 -
That being the case, considering Matrix Brandy uses 64-bit floats, if the allocation is high enough that the address cannot be accurately stored in a float then it's being sent off in a wrong direction (but not far off enough to throw a SEGV error, reported as Address exception).
A while back I had an attempt at making a variant type, but failed miserably. So it looks like I need to revisit this, or deal with some kind of "virtual 32-bit"addressing (where, if the main workspace address isn't 32-bit clean, the top 32-bits are OR'd with any address that is 32-bit clean - but that sounds like it can get very hairy very quickly. I would rather not go down that road.)0 -
In my BASICs I'm still getting a heap address that can be precisely represented by a 64-bit double (i.e. a 53-bit mantissa) so that's not a likely cause. If you can remind me how to print 64-bit hexadecimal numbers I'll tell you what it is in Brandy.That being the case, considering Matrix Brandy uses 64-bit floats, if the allocation is high enough that the address cannot be accurately stored in a float then it's being sent off in a wrong direction (but not far off enough to throw a SEGV error, reported as Address exception).
0 -
SYS"Brandy_Hex64",10
-
-
I can replicate this on Win11 (my work laptop) - and more usefully, on Linux when I use DIM HIMEM (which just uses malloc() to get some off-heap space). Time to do some digging...0
-
-
And there, sorted. I've manually kicked off a rebuild of the nightlies for Win32, Win64, EL7-9 and Fedora 38-39.
It was something missed from way back when, when I first got Brandy running on 64-bit, where it converted the float containing the address to a 32-bit int where it should have been updated to a 64-bit int.
(I wonder how many more of those are lurking...)0 -
That suggests to me a weakness in the original code, because surely the float should never have been converted to a 32-bit int (or indeed an int of any specific size) but to an intptr_t which is an int that is the same size as a pointer. Had that been the case the code shouldn't have needed any alteration.it converted the float containing the address to a 32-bit int where it should have been updated to a 64-bit int.
0 -
The "Number out of range" error was raised by the TOINT() function in the code, that raises the error if the conversion from float to int couldn't be done. The version in the manual nightly just uses TOINT64() which does similar checks, but for 64-bit.
I'm making further tweaks that uses a new function that will use TOINT() on a 32-bit build, and TOINT64() on 64-bit. This will be in the automated nightly builds.
Also, seems like I need to pull in another header to get intptr_t (stdint.h), elsewhere in the code I've been (ab?)using size_t to have similar effect.0 -
Ah, I've never heard of the TOINT() function (not surprising given my minimal knowledge of C).The "Number out of range" error was raised by the TOINT() function in the code
If it's the conversion from a float to an integer that's raising the error, it surprises me that !b works, because surely that has to do the same conversion as b!0?
Me too, but size_t is unsigned so strictly it's uintptr_t that it's equivalent to (in all modern CPU architectures).I've been (ab?)using size_t to have similar effect.
0 -
TOINT() and similar are internal functions within the Matrix Brandy source code that do range checking prior to actually making the conversion, so it can raise an error. It's not a standard libc function.
I've made some further tweaks, making Matrix Brandy display a warning (but not throw an error) if trying to bung a 64-bit int into a 64-bit float would cause loss of precision, and I noticed that bit shifting wasn't behaving entirely correctly - and in fixing it simplified the code a lot. (If you want to have a play, unlike yours where *HEX 64 also seems to make bit shifting work at 64 bits, Matrix Brandy has a separate twiddle for this - SYS"Brandy_BitShift64",1 which only affects left shifting, as right shifting does the "right thing" in both 32 and 64-bit space)0 -
In the version of Matrix Brandy I have, (unsigned) right-shifts are affected, as I would expect. Is that what you meant, or have you made a change which means that even unsigned right-shifts are no longer affected?SYS"Brandy_BitShift64",1 which only affects left shifting, as right shifting does the "right thing" in both 32 and 64-bit space
If your changes will be in tonight's Windows Nightly I'll be able to try it myself, I know.
0 -
The changes will be in tonight's nightlies. I have a script that fires at 1am every morning to run a build if a change is detected.
After this, right shifts won't be affected by the twiddle as they will do the "right thing" on 32-bit values even in 64-bit space (and a typed variable can force its hand to treat a small number as 64-bit for example). Left-shifts will look at the twiddle to determine whether the shifts happen in 32-bit space or 64-but space.0 -
Not according to my tests - 64-bit unsigned right-shift seems to be completely broken now, as you can see from the screenshot below (prints 7FFFFFFF when it should be 7FFFFFFFFFFFFFFF).After this, right shifts won't be affected by the twiddle as they will do the "right thing" on 32-bit values even in 64-bit space
I'm puzzled that you should have, superficially at least, broken something that was previously working correctly; and even more concerned that despite this 'breaking change' the version number and date are the same as in the earlier release (version 1.22.15 on 21 Mar 2023).
If this is the conventional major.minor.patch Semantic Versioning scheme I would expect the patch field to change with every functional modification and probably the minor field with such a serious bug being introduced.
0 -
These are nightly builds, rather than releases. Fair enough I should perhaps tag the title bar or the startup message pointing out it's a development build.
Edit: This is now in place (for the Windows builds, working on the Linux scripts now), and the changes below are included in a manual run I've pushed.
The issue you ran into was a result of ensuring that:>A%=-1 >PRINT ~A% >>> 1 7FFFFFFF >_
would not break existing programs working in 32-bit space.
The fix for this, to ensure everything plays nicely is that if Hex64 is set, then TRUE returns (int64)-1 instead of (int32)-1. To ensure the shift doesn't pick up a sign bit, the logical shift right works in unsigned int64 space, and known 32-bit integers from % variables thus are cast to unsigned before being promoted to 64-bit therefore allowing the above to still work.
With the latest change, this now works:>SYS"Brandy_Hex64",1 >SYS"Brandy_BitShift64",1 >A%=-1: A%%=-1 >PRINT ~A% ' ~A%% FFFFFFFFFFFFFFFF FFFFFFFFFFFFFFFF >PRINT ~A% >>> 1 ' ~A%% >>> 1 7FFFFFFF 7FFFFFFFFFFFFFFF >PRINT ~TRUE >>> 1 7FFFFFFFFFFFFFFF >SYS"Brandy_Hex64",0 >SYS"Brandy_BitShift64",0 >PRINT ~TRUE ' ~TRUE >>> 1 FFFFFFFF 7FFFFFFF >_
0 -
If the version number and date in the startup message are meaningless, which would seem to be the case, then it would be far better to remove them altogether or replace them with xx or something.Fair enough I should perhaps tag the title bar or the startup message pointing out it's a development build.
But that worked perfectly prior to yesterday's change, as far as I'm aware. There wasn't a problem that needed 'fixing', indeed I think Matrix Brandy worked identically to my BASICs (apart from the way of changing the hex and shift functionality). Here's what my BASICs produce with that same code:The issue you ran into was a result of ensuring that:>A%=-1 >PRINT ~A% >>> 1 7FFFFFFF >_
would not break existing programs working in 32-bit space.>A%=-1 >PRINT ~A% >>> 1 7FFFFFFF >
But does this work:With the latest change, this now works...SYS"Brandy_BitShift64",1 PRINT TRUE >>> 1
This should print 9.22337204E18 or thereabouts. From your description of the latest change it sounds as though it might print 2.14748365E9 which of course would be completely wrong.
I don't understand the motivation for any change having been made at all, if it wasn't broken why fix it?0 -
I'm trying to remember exactly what it was that I spotted the shifting not working entirely correctly (and failing...sigh, I wish I had put an example in the comment above), and the code was rather convoluted trying to cover each type of input differently and was hard to read. The rework has simplified the code a lot, and the issues with TRUE came about because it was pushed to the stack as a 32-bit int.
I have fixed the issue you raised there (and rebuilt), so TRUE is now 64-bit if either twiddle is set. Perhaps I should retire BitShift64 and pivot only on Hex64 (and turn Brandy_BitShift64 as a deprecated alias to Brandy_Hex64).0 -
All these ad-hoc tweaks seem highly dangerous and undesirable to me. For example if your description of the latest iteration is taken literally it would mean that:I have fixed the issue you raised there (and rebuilt), so TRUE is now 64-bit if either twiddle is set.SYS "Brandy_Hex64", 1 PRINT TRUE >>> 1
would print 9.22337204E18 when of course it should print 2.14748365E9 (hexadecimal notation isn't used at all in that code so the SYS should have no effect).
Personally I would recommend reverting yesterday's changes; code elegance is all very well but ugly code that works is better than elegant code that doesn't work. Whatever happened to code 'design' or 'software engineering'? It seems to have been abandoned on this occasion.
0 -
You're right, once again it shows that I'm not a formal software engineer.
I have backed out the bit shift changes (and the switching how TRUE returns). I have left in the changes made to deal with resolving addresses from a float (which started this thread), and a switchable warning if an int64 can't fit accurately into a float. (It's switched on "strict" mode, using -strict as a parameter to brandy, or SYS"Brandy_Strict",1 at runtime. There were a couple of builds earlier that triggered it regardless of strict mode.)0 -
I'd be surprised if you ever see this. A 64-bit double can (I think) hold a 54-bit signed integer precisely, which is 16 Petabytes (or 16,000 Terabytes or 16 million Gigabytes). Although an Operating System could, in principle, allocate virtual memory anywhere in the 64-bit address space, in practice it is usually allocated either from the bottom up or from the top down. So the chance of allocating an address more than 8 Petabytes from the bottom or 8 Petabytes from the top is somewhat slim!a switchable warning if an int64 can't fit accurately into a float.0 -
In other use cases aside from addresses it's fairly easy to trigger.
For example:SYS"Brandy_BitShift64",1 SYS"Brandy_Strict",1 PRINT TRUE >>> 1
(Printing as a decimal makes it get handled as a float)
Strict mode has a few other effects.
Without it, LOCAL with no parameters in a function or procedure is a no-op (as per Acorn). (I discovered by accident at an ABUG meet last year, when it raised an error, meaning an old program wouldn't run. Arguably, that empty LOCAL is a bug in the BASIC program, but if that didn't stop it running correctly on the BBC Micro, then it shouldn't stop running on Matrix Brandy). With strict mode on, a Syntax error is raised, as your BASICs do. Other effects are that warnings are handled as errors, stopping the program, and MOS functions that are not supported in text mode raise an Unsupported error, without strict mode these are no-ops. (Attempts to use an assembler will raise an Unsupported error regardless of this switch.)0 -
There will always be a 'loss of precision' when PRINT attempts to output an integer but, because it cannot be represented precisely in the current @% mode (or at all), it gets displayed in E-format. So for example with the default @%=&90A mode PRINT 12345654321 outputs 1.23456543E10. Does that trigger a warning? I sincerely hope not!In other use cases aside from addresses it's fairly easy to trigger.
For example:SYS"Brandy_BitShift64",1 SYS"Brandy_Strict",1 PRINT TRUE >>> 1
So neither should 'PRINT 2^300' or 'PRINT TRUE >>> 1' or anything else that is output by PRINT, the 'loss of precision' is inevitable and happens in every version of BBC BASIC (indeed pretty much every language). Issuing a warning, especially one that happens when a 64-bit integer cannot be printed precisely but not when a 32-bit integer or rational value can't, is plain wrong in my opinion.
P.S. This crappy forum is freezing on me regularly today (and before you ask Adblock Plus is still disabled on this site).
0 -
-
Matrix Brandy has to evaluate 2^63 as a float - as such, since its floats are 64-bit, this actually didn't work as 2^63 - 1 doesn't have enough precision left in float64 space. I've sorted this out for subtraction of floats by temporarily promoting the value internally to long double (I know this won't help on ARM, but won't make things worse there). Then, it returns the result, checking if it can be returned as a 32-bit int, or a 64-bit int, and a 64-bit float if it can't be represented as either int.
A possible useful project for the future may be to see if I can get Brandy to use "long double" as its float representation throughout. The catch here is, BASIC VI is 64-bit floats (which is why I retagged Matrix Brandy as a BASIC VI interpreter, upstream tagged it as BASIC V), and the standard BASIC V uses the traditional 40-bit length as used on the BBC Micro.0 -
OK, that's a fair comment, and the same is true of my BASICs when running on ARM CPUs (which have only 64-bit floats). I should have formulated the example better, say:Matrix Brandy has to evaluate 2^63 as a floata%% = 2^62-1+2^62 @%=&1414 PRINT a%% 9223372036854775807
That runs fine even on my ARM editions so presumably also does in Matrix Brandy. It's an entirely integer operation not requiring conversion to float at any point, and not requiring variant numeric variables (which I know Matrix Brandy will probably never have, which is fine).
My point stands that had your claim that "printing as a decimal makes it get handled as a float" been true even that example wouldn't work. It would be very peculiar (and unlike any programming language I know) to associate 'hexadecimal' with 'integer' and 'decimal' with 'float'. Hex versus decimal is a matter of number base (16 or 10), quite independent from integer versus float; you can represent any integer as hex or decimal.
I know 'upstream Brandy' does some peculiar things but I hope that isn't one of them, it would give mathematicians conniptions!
0
