Matrix Brandy V1.23.0 released

Soruk · February 6

I have today released version 1.23.0 of Matrix Brandy.

Changes include:

- System: Add '-tek' (or -t) option to enable Tek graphics for 'tbrandy' on compatible platforms. Equivalent to issuing SYS"Brandy_TekEnabled",1
- System: Strict mode can now be set programmatically with SYS"Brandy_Strict".
- System: Strip quotes from filenames in *LOAD and *SAVE
- System: Implemented OSWORDs 1 and 2
- System: Fix file name bug in OSFILE.
- System: Filename translation between RISC OS and Linux for OPENIN, OPENOUT and OPENUP.
- System: Added "-nofull" command line option, to disable switching into full-screen mode (for questionable hardware, e.g. Poulsbo chipsets)
- System: Do not hang on text build startup when console doesn't handle the ANSI sequence to report the cursor location (e.g. Minix3 console), instead time out after 0.5 seconds.
- MOS: *Refresh, *WinTitle and *FullScreen are now recognised on RISC OS builds, but do nothing. (Previously, they raised a "Not Found" error.) This is to allow programs written targeting other platforms a better chance to run correctly under RISC OS without further modification.
- MOS: OSBYTEs 42-44 deprecated, use 163,2-4 instead.
- BASIC: LOCAL with no parameters now does nothing (as per Acorn) instead of reporting an error. This new behaviour is in place on a DEFAULT_IGNORE build unless -strict is set, or if -ignore is set. This can also be toggled at runtime with SYS"Brandy_Strict".
- BASIC: If Strict option is set, warnings are handled as fatal errors if a program is running.
- BASIC: Limit the recursion depth of RUN.
- BASIC: Don't choke if a numeric DATA element contains a comma (e.g. multi-dimensional array member or FN call)
- BASIC: Clear variable stubs if a DIM fails, and after CLEAR HIMEM.
- BASIC: Indirection with floats containing addresses > 32 bits now work.
- BASIC: Print a warning if a conversion from int64 to float loses precision.
- BASIC: Fix bugs in Exponent format number printing. Graphics: Fixed multiple problems with VDU24, RECTANGLE FILL, MOUSE and the plot_pixel() primitive, primarily to do with the oddball MODE 22. Thanks to Michael Fairbank for identifying and assisting with fixing this.
- Graphics: Fixed a cursor placement bug for VDU31 (PRINT TAB(x,y)) in VDU 5 mode.

As of this release, the git repository master branch will track releases. Development will be found in the dev-main branch.

Downloads are available, as usual, from the website.

[Richard Russell] · February 10

Soruk wrote: »

- BASIC: Limit the recursion depth of RUN.

Can you explain that one please?

A link to the download would have been very nice!

Soruk · February 13

This is as a result of RUN calling a function in the C code, and a program that calls RUN could exhaust the C stack if called recursively.

[Richard Russell] · February 13

Soruk wrote: »

This is as a result of RUN calling a function in the C code, and a program that calls RUN could exhaust the C stack if called recursively.

Ah, is this an extension to RUN that I'm not aware of, or did you mean *RUN?

Soruk · February 13

No, it's just the way it works in the C code, that RUN is implemented as a C function that is called, thus it's called like a PROC rather than a GOTO in the C layer. So, if a program calls RUN within itself it can lead to C stack exhaustion or the process running out of memory, so I set a recursion limit to prevent it crashing the interpreter.

[Richard Russell] · February 13

Soruk wrote: »

RUN is implemented as a C function that is called, thus it's called like a PROC rather than a GOTO in the C layer. So, if a program calls RUN within itself it can lead to C stack exhaustion or the process running out of memory

Interesting. Why is RUN conceptually any different from CLEAR : GOTO 1 (or whatever the first line number in the program is) or would that code have the same complication? Is CHAIN affected as well (I'm guessing it probably is given the similarity of the statements)?

If RUN is somehow different from CLEAR : GOTO and CHAIN I feel I must be missing something fundamental, which is rather worrying because it might mean that my interpreters are affected in a similar way without me ever realising it! Running this code doesn't cause any obvious issue in my coded-in-C interpreter:

      A% += 1
      PRINT A%
      RUN

Soruk · February 13

I'll have to take a closer look at this, it's code inherited from upstream that I haven't had to touch (beside tracking the recursion level). The limit is pretty high, 512 levels, and a program that refuses that much is likely to have gone wrong.

The issue is also likely to affect CHAIN, again I need to take a closer look.

Edit: Recurses, not refuses. Autocorrupt, typing on my phone.

[Richard Russell] · February 13

Soruk wrote: »

The limit is pretty high, 512 levels, and a program that refuses that much is likely to have gone wrong.

I commonly use RUN in my programs, typically after a window resize when I want to reinitialise everything from scratch for the new size. Because one might get a large number of window resize notifications whilst dragging a corner with the mouse, dozens of RUNs isn't that unlikely:

      ON MOVE IF @msg% = 5 RUN ELSE RETURN : REM Resize

If CHAIN is also affected, in the context of an IDE which uses CHAIN to run the 'spawned' programs (my touchide.bbc is like that) there is no limit to the number of CHAINs one might get in a 'session'.

Soruk · February 16

While I am still to look at CHAIN, RUN is now resolved - it now uses setjmp/longjmp if the program is running to reset the C stack if RUN is called while a program is running. This ought to mean the repetition depth is unlimited. (After all, it isn't actual recursion as a program can't return from a RUN.)

[Richard Russell] · February 16

Soruk wrote: »

RUN is now resolved - it now uses setjmp/longjmp if the program is running to reset the C stack if RUN is called while a program is running.

I'm still a little confused. After your change is RUN still different from CLEAR:RESTORE:GOTO 10 (where line 10 is the first line of the program) and if so why? I'm pretty sure I've always assumed the two to have identical* effect, and if they don't I'd put that in the category of 'surprising'.

* Except perhaps for RUN also clearing the BASIC stack.

Soruk · February 16

I looked at that approach, but GOTO <nonexistent line number> causes an error (as it does on the BBC, I think the Spectrum jumps to the next valid line). So if the "program running" flag is set, a RUN does a longjmp() back to the initialisation point, which also resets the stack - so a program constantly calling RUN won't cause the stack to grow.

I've been doing more work on the stack issues though, recursion of FN causes both the BASIC stack AND the C stack to grow, so I've changed the threading model a bit because pthreads allows me to set a custom stack size for a thread, and the artificial threading limit is based on a function of the supplied stack size (itself based on workspace size). Surprisingly this even works on Minix.

[Richard Russell] · February 16

Soruk wrote: »

I looked at that approach, but GOTO <nonexistent line number> causes an error (as it does on the BBC

Sorry if it wasn't obvious, but that was 'conceptual' code, not actual code! Of course one cannot actually say GOTO <first line in program> - there's no syntax for that in BBC BASIC - my point was that I would expect RUN to have the same effect if there was! :-)

I've been doing more work on the stack issues though, recursion of FN causes both the BASIC stack AND the C stack to grow

Why do you care? I would expect using RUN in a FN to blow up because of unconstrained growth of the stack - it does in mine too - I don't consider that to be a fault of the interpreter but of the BASIC programmer!

In general I always follow the recommendation of Raymond Chen, the famous Microsoft blogger and Windows expert, who says you shouldn't attempt to trap running out of memory but rather you should let the program crash. It's safer and it's faster.

The nature of BBC BASIC is that you can crash the interpreter using BASIC code. That's not a fault that needs to be corrected. Spend your valuable time adding new features and fixing real bugs!

Soruk · February 16

It's not a RUN in an FN, more like something contrived like this:

   10 A%=0
   20 PRINT FNr
   30 END
  100 DEFFNr
  110 A%=A%+1
  120 PRINT A%
  130 =FNr

When the C stack is breached, you get a segmentation fault error, so I've been trying to prevent this. Of course it's not 100% possible to block this, I've been looking to see how I can deal with the stack and set a sensible recursion limit based on the stack size (which is now a known value).

An error will happen, what I am trying to avoid is the interpreter crashing out.

[Richard Russell] · February 16

Soruk wrote: »

It's not a RUN in an FN, more like something contrived like this:

Yes, in my BASIC the effect is the same: whether it's by RUN in FN or by FN in FN the recursion will result in a crash owing to the C stack blowing up.

When the C stack is breached, you get a segmentation fault error, so I've been trying to prevent this.

I understand, but why bother, especially as you can never fully succeed? If it was a sandboxed language in which it mustn't be possible to crash the interpreter using BASIC code then fair enough, but BBC BASIC has low-level features which make that impractical.

For most of its history it has been impossible to protect BBC BASIC from crashing as a result of !<random address> or USR(<random address>) etc. and everybody accepts that as a price worth paying for the benefits of those low-level features.

You seem to be trying to leverage the hardware memory management features of modern CPUs to avoid those crashes, but in my opinion your efforts are misguided. Those same hardware features already mean that the crash just takes out one process, not the entire machine, so is harmless.

[Richard Russell] · February 16

This is Raymond Chen's original article, in which he mentions stack growth.

Incidentally I'm not sure what the default stack size is for 64-bit GCC and Clang builds, do you know (it used to be a few megabytes for 32-bit builds I think)? It's usually possible to override that with a linker flag, but I don't do that in any of my builds.

Soruk · February 16

In Linux, the stack limit tends to be set with 'ulimit -s'.

However, pthreads allow you to set thread attributes including the thread stack size. Ever since I went multi-threaded I had the interpreter in a separate thread to the display (which must be the top level in SDL 1.2), I just switched from using SDL's threading functions to pthreads - which actually simplified the code as I needed pthreads for the text-mode builds anyway. Thus, by a bit of trial and error I've got a stack size that is related to the workspace size. Previously, especially in Windows I was getting a FN recursion limit barely bigger than I was seeing in HIBASIC 4.30 on the BBC Master, despite a workspace size of 64MBytes, versus the Master's 45.5K in HIBASIC! Even ARM BBC BASIC VI was giving me around 27000 in the default 640K workspace. Now I am getting a depth of about 100,000 in the 64MB workspace size. Matrix Brandy is clearly not as efficient as the ARM or 6502 BASICs in their use of BASIC workspace, when I tweak the algorithm to give way more stack than is needed (so the BASIC workspace is exhausted first), a 640K workspace gives me a recursion depth of just 1487, and with 46K workspace(!) I get only 105. The Master got about 1800.

RISC OS was the odd one out here, as I'm not using a threading model as I don't need to worry about graphics updates or the centisecond timer (as RISC OS provides both of these in a directly compatible fashion). The stack, at least on the UnixLib build grows as needed, but has a hard limit when the process size reaches the WimpSlot maximum of 28640K. So I've worked out the limit there and hardcoded a cap that's a bit below that, but of course on a machine with not so much memory it's still possible to crash it.

[Richard Russell] · February 16

Soruk wrote: »

I just switched from using SDL's threading functions to pthreads

SDL2 has SDL_CreateThreadWithStackSize which I don't use at the moment but could experiment with. However unless I can increase the stack size on every platform I support, there's no point in increasing it on any of them. Some (e.g. Android and iOS) may well have relatively small limits imposed at run-time because of the nature of the OS.

Stack size has never been an issue for any of my programs, and nor has anybody else ever reported it being so. I can't imagine any reasonable circumstances when a recursion depth of several thousands, let alone tens of thousands, would be required.

When running on the Raspberry Pi Pico, with a total RAM (for everything) of only about 256 Kbytes, the limit is much smaller but again that has rarely been an issue. Only my Sudoku solver program has been known to run out of stack on that platform.

[Richard Russell] · February 16

Richard_Russell wrote: »

Some (e.g. Android and iOS) may well have relatively small limits imposed at run-time because of the nature of the OS.

I've just carried out a little experiment using SDL_CreateThreadWithStackSize(). On desktop platforms I can indeed increase the maximum recursion depth that way but in the in-browser (Emscripten) edition it makes no difference, and the maximum recursion depth remains at a few hundred.

Soruk · February 16

That's a similar approach my code takes - if the stack can be increased, great. If it can't, oh well, we'll just make do with what we have.

[Richard Russell] · February 16

Richard_Russell wrote: »

the maximum recursion depth remains at a few hundred.

I'm going to set you a little challenge, list a program which does something useful and is best implemented using recursion to more than 100 levels deep.

Soruk · February 16

I'm not sure if a flood fill counts as useful...

    5REM>FFbasic
   10MODE 19: REM 640x512 version of MODE 1
   15O%=2: REM Pixel interval, set to 4 for MODE 1
   20FOR X%=0 TO 1280 STEP (O%*2)
   30MOVE X%,0: DRAW X%,1280
   40MOVE 0,X%: DRAW 1280,X%
   50NEXT
   60GCOL 0,1
   70GCOL 0,131
   80PLOT &85,0,0
   85WAIT 100
   90GCOL0,2
  100PROCfill(640,512)
  110PRINT"Max depth: ";M%;" Depth at end: ";D%
  120END
  200DEFPROCfill(x%,y%)
  210OFF
  220D%=0:M%=0
  230C%=POINT(x%,y%)
  240PROCfill2(x%,y%)
  250ON
  260ENDPROC
  270DEFPROCfill2(x%,y%)
  280IF x%<0 OR y%<0 OR x%>1279 OR y%>1023 THEN D%-=1: ENDPROC
  290POINT x%,y%
  300D%+=1
  310IFD%>M% THEN M%=D%
  320IF POINT(x%-O%,y%)=C% PROCfill2(x%-O%,y%)
  330IF POINT(x%,y%-O%)=C% PROCfill2(x%,y%-O%)
  340IF POINT(x%+O%,y%)=C% PROCfill2(x%+O%,y%)
  350IF POINT(x%,y%+O%)=C% PROCfill2(x%,y%+O%)
  360D%-=1
  370ENDPROC

[Richard Russell] · February 16

Soruk wrote: »

I'm not sure if a flood fill counts as useful...

I'm sure it does, but I thought it had long been established that a naïve recursive solution is not the best way to perform a flood fill? For example here:

https://codeheir.com/2022/08/21/comparing-flood-fill-algorithms-in-javascript/

if the stack can be increased, great. If it can't, oh well, we'll just make do with what we have.

My approach is the exact opposite, I would want to limit the maximum recursion depth, on those platforms which can do more, to no greater than what is imposed by the one with the smallest stack.

My reasoning is this. If, for example, one wants to develop an Android app in BBC BASIC the development is likely to be done on a Windows or MacOS desktop, and only when the code has been thoroughly tested in that environment will it be tried on Android.

The last thing you want to discover at that late stage is that the algorithm you chose needs a stack greater than is available in Android! Far better to discover that earlier, when testing on the PC, so you change your approach then.

[Richard Russell] · February 16

Soruk wrote: »

I'm not sure if a flood fill counts as useful...

Incidentally I'm sure you don't need me to point out that the code you listed is not recursive as far as the C stack is concerned, as there are no user-defined functions (FN) only PROCs. The maximum depth will be determined by the heap size only.

Generally speaking if the C stack is the limiting factor the likelihood is that the code can be rejigged to use a PROC with a RETURNed parameter rather than an FN. Here's a comparison I made using BBCSDL:

      PRINT FNr(0)
      END
      DEF FNr(A%)
      PRINT A%
      A% += 1
      = FNr(A%)

Bombed out after 575 recursions because of C stack overflow.

      PROCr(A%) : PRINT A%
      END
      DEF PROCr(RETURN A%)
      PRINT A%
      A% += 1
      PROCr(A%)
      ENDPROC

Bombed out after 466,000 recursions because of BASIC stack overflow.

Soruk · February 16

Richard_Russell wrote: »

Soruk wrote: »

I'm not sure if a flood fill counts as useful...

I'm sure it does, but I thought it had long been established that a naïve recursive solution is not the best way to perform a flood fill? For example here:

https://codeheir.com/2022/08/21/comparing-flood-fill-algorithms-in-javascript/

I'll take a closer look at this, and sure, I know it's hardly a good algorithm, it's just one that I threw together quickly and did the job. (I ported the code to BASIC 4 on my Master, and it rapidly runs out of space even in HIBASIC.) Conversely, while slow, the Master's internal fill can fill that pattern (which I agree, is hellish for a recursive algorithm), slowly, but it completes - even with the 2nd processor disabled. I confess, the internal algorithm that implements PLOT &85 is a similarly lame recursive algorithm, but it relies on modern kit having far more space than the old 1980s systems.

I guess my first step would be to try to implement some of these in BASIC, then once I fully understand how they work, port them to C to improve the PLOT &85 handler.

[Richard Russell] · February 17

Soruk wrote: »

Conversely, while slow, the Master's internal fill can fill that pattern

Even without seeing the code one can guess the flood fill algorithm probably used in the Master, because of the 'horizontal fill primitives' present in the original BBC Micro MOS: PLOT 72-79, PLOT 88-95, PLOT 104-111, PLOT 120-127.

I've never implemented them fully (particularly in respect of the way they are supposed to move the graphics cursor) but I expect you do, in which case you should be able to make a respectable flood fill using them; I think you'll find it's the 'span' algorithm at the Javascript link.

The span algorithm performed the best of the bunch in those comparative tests, so it seems Acorn made a good choice of fill primitives in the BBC Micro. I can't say that I've ever seen a BASIC flood fill using those primitives, but it's what they were intended for.

Matrix Brandy V1.23.0 released

Comments

Categories