Mysterious failure in MacOS

[Richard Russell]

As this forum is the closest to being a "developers'" forum of any that I use, I'm hoping somebody here can shed some light on what (to me) is a mysterious and inexplicable failure in MacOS.

The code concerned is this, where socket is a UDP socket:

    result = fcntl(socket, F_SETFL, O_NONBLOCK);

This code has the expected effect in every variety of Linux I've tried (including Raspberry Pi OS): it sets the socket to non-blocking and returns a value other than -1 to indicate success.

But in MacOS it returns a value other than -1 (which should indicate success) but does not set the socket to non-blocking and does not set the O_NONBLOCK flag, as checked using a F_GETFL call to read the current flags. This makes no sense to me.

[Richard Russell]

I'm wondering if this could be another Apple 'security' measure. Have they perhaps decided that making a socket non-blocking is 'dangerous' and silently stopped you doing it. It doesn't seem very likely, but I did find a report from the Malwarebytes people which listed fcntl() as a function which could be vetoed in recent versions of MacOS.

Soruk

It sounds like they're trying to turn your computer into a locked-down tablet with built-in keyboard.

[Richard Russell]

Soruk wrote: »

It sounds like they're trying to turn your computer into a locked-down tablet with built-in keyboard.

Typical of Apple! But I'm not sure the conspiracy theory holds water in this case, because you can still achieve an equivalent (if less efficient) functionality using select() or poll(), so it just seems that for some strange reason MacOS doesn't support non-blocking sockets, whereas Windows and Linux do.

I can't find that explicitly documented anywhere, but there are a few reports of code which relies on non-blocking sockets failing when ported to MacOS. How do you achieve non-blocking network access in Matrix Brandy?

Soruk

I've not done anything special to support it, and to be fair I don't have the latest version of MacOS running on my VM server. I have High Sierra, Catalina and Big Sur. I attempted to upgrade beyond Big Sur but it refused to boot, thank goodness for VM snapshots so I could roll back. I don't own any physical overpriced junk Apple kit.

[Richard Russell]

Soruk wrote: »

I've not done anything special to support it

I'm pretty sure sockets are blocking, by default, on all platforms. So if you are doing 'nothing special' then perhaps the networking in Matrix Brandy is blocking, currently. Obviously that would be highly limiting, since it wouldn't allow a BASIC program to do something useful while waiting for incoming data, but if it's the only functionality you advertise then so be it.

Soruk

One other point, Matrix Brandy only supports TCP sockets, and then only as a client, not a server.

[Richard Russell]

Soruk wrote: »

Matrix Brandy only supports TCP sockets, and then only as a client, not a server.

The MacOS issue affects TCP and UDP sockets equally. I initially noticed it with UDP sockets, but when I checked TCP sockets they couldn't be set to non-blocking either.

When you say "only as a client, not a server" does that mean you can't set up a point-to-point network connection between two instances of Matrix Brandy?

Soruk

In my net code, I blindly set O_NONBLOCK when opening my network socket, which is done in BASIC by extending OPENUP(). I guess opening a server socket could be done by extending OPENOUT? But yes, right now, two Matrix Brandy instances can't talk to each other over a network connection.

I'll have a bit of a play later tonight to see the effect of not setting that either in Linux or MacOS, as I'm pretty sure my Telstar client relies on the non-blocking nature of the socket, and it has so far run fine under my virtual Hackintosh.

[Richard Russell]

Soruk wrote: »

In my net code, I blindly set O_NONBLOCK when opening my network socket

I think that's highly likely not to work in MacOS. Are you using fcntl() or ioctl() to set it non-blocking?

Soruk

Richard_Russell wrote: »

Soruk wrote: »

In my net code, I blindly set O_NONBLOCK when opening my network socket

I think that's highly likely not to work in MacOS. Are you using fcntl() or ioctl() to set it non-blocking?

I'm using fcntl() on all platforms other than Windows/MinGW, where I'm using ioctl(). However, probably in an example of bad programming, I'm ignoring the result and blindly calling it! Though, with a fprintf() wrapping this, I can see on Catalina it returns 0.

For what it's worth, my Telstar client relies on the underlying ability of my network client to request a block, and not just hang until it gets one, but instead offer two negative return codes via BGET#; -1 if no data waiting in the buffer, the client code failed to retrieve more data but the remote end hasn't closed the connection, and -2 if the remote end has closed the connection and the buffer is exhausted.

[Richard Russell]

Soruk wrote: »

However, probably in an example of bad programming, I'm ignoring the result

It wouldn't make any difference, since it returns zero (success) on the Mac anyway.

The conclusion I draw from my tests is that although O_NONBLOCK is defined on the Mac (interestingly, with a different value from that on Linux, despite being 'standardised' in POSIX) it doesn't work for sockets, which always block.

If that's right, I assume that the reason for fcntl() returning 'success' is because failure to set a flag that doesn't exist doesn't count as an 'error', only failure to set a flag that does exist would be an error. Not necessarily helpful, but the only explanation I can think of.

For what it's worth, my Telstar client relies on the underlying ability of my network client to request a block, and not just hang until it gets one

So does your Telstar client seem to work on your emulated Mac? If it does I'm at a loss to explain what's going on. Maybe I made some stupid mistake, but I spent hours trying different ways of making a socket non-blocking and never found one.

Soruk

Yep, it seems to work fine, so it never occurred to me there might be an issue with it!

My net driver is here --> https://github.com/stardot/MatrixBrandy/blob/master/src/net.c
the fcntl() call is on line 308. The only platforms I don't use fcntl() are MinGW where I have to use ioctlsocket(), and RISC OS (SharedCLibrary target) where I have to use socketioctl().

[Richard Russell]

Soruk wrote: »

Yep, it seems to work fine, so it never occurred to me there might be an issue with it!

OK, so there must be non-blocking sockets in MacOS and I must have done something stupid to conclude that there aren't. I wonder if it might have something to do with fcntl() being a variadic function; I haven't had any difficulty calling fcntl() from BASIC code in Linux (or indeed sprintf() in Windows), but perhaps MacOS handles variadic functions differently.

My net driver is here --> https://github.com/stardot/MatrixBrandy/blob/master/src/net.c
the fcntl() call is on line 308.

Everything I've found in my searches (and believe me I searched high and low for why I couldn't succeed in creating a non-blocking socket) says that you must never just set O_NONBLOCK (because that has the side-effect of resetting all the other flags) but instead 'or' it with the existing flags:

fcntl(mysocket, F_SETFL, O_NONBLOCK | fcntl(mysocket, F_GETFL));

I'm pretty sure that in MacOS there is at least one other flag set by default after opening the socket (I think it might be FWRITE), because part of my testing involved reading the current flags.

Soruk

You're absolutely right about fcntl(), I was misusing it and just by sheer dumb luck it was working as I wanted it to. So that is now fixed and checked in (and I was half expecting my Telstar client to stop working correctly on my virtual Mac, but no, it still works.)