LuaSocket, Mac vs. Linux - further info...

From my previous blog post, see the following

https://stackoverflow.com/questions/51911564/cannot-bind-socket-to-port-in-both-ipv4-and-ipv6

It seems likely this is the problem... although I haven't tested it yet.

A fragment:

IPV6_V6ONLY (since Linux 2.4.21 and 2.6) If this flag is set to true (nonzero), then the socket is restricted to sending and receiving IPv6 packets only. In this case, an IPv4 and an IPv6 application can bind to a single port at the same time. If this flag is set to false (zero), then the socket can be used to send and receive packets to and from an IPv6 address or an IPv4-mapped IPv6 address.

The argument is a pointer to a boolean value in an integer.

The default value for this flag is defined by the contents of the file /proc/sys/net/ipv6/bindv6only. The default value for that file is 0 (false).

Taken from man 7 ipv6

 

 

Also see: 

https://stackoverflow.com/questions/1618240/how-to-support-both-ipv4-and-ipv6-connections

The best approach is to create an IPv6 server socket that can also accept IPv4 connections. To do so, create a regular IPv6 socket, turn off the socket option IPV6_V6ONLY, bind it to the "any" address, and start receiving. IPv4 addresses will be presented as IPv6 addresses, in the IPv4-mapped format.
The major difference across systems is whether IPV6_V6ONLY is a) available, and b) turned on or off by default. It is turned off by default on Linux (i.e. allowing dual-stack sockets without setsockopt), and is turned on on most other systems.
In addition, the IPv6 stack on Windows XP doesn't support that option. In these cases, you will need to create two separate server sockets, and place them into select or into multiple threads.

NOTE: Then commenter's mention IPV6_V6ONLY is off by default on Linux is wrong - it depends on the configuration on the distribution.

  

LuaSocket connection problems on Mac OS Catalina 10.15

UPDATE: If you are interested in the below, then you should check out the update.

Our game runs as server or client, but we had a problem where recently, the game on a Mac stopped connecting to the same localhost Mac server (for testing).

We are using copas which calls LuaSocket, but I tested without and eliminated that.  

So:

• Linux Desktop Client to Linux Desktop Server - works fine
• Mac Desktop Client to Linux on Raspberry Pi test server - works fine
• Mac Desktop Client to same Mac Desktop server - doesn't work.

The error in the client (returned by the following code) is 'connection refused'. 

local success, err = copas.connect(master_socket, ServerIPAddress, ServerPort)

Tested the Mac Desktop Client to connect to port 22... no connection refused from this line. (I have ssh enabled on this machine).

Tested with two sets of Python 3 code as the server:

sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM) server_address = ('localhost', 25561) sock.bind(server_address) sock.listen(1) connection, client_address = sock.accept()

and

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server_address = ('localhost', 25561) sock.bind(server_address) sock.listen(1) connection, client_address = sock.accept()

 

Since it connected to the first one, and not the second one, it leads me to believe it's trying to connect to IPv6 and failing to the server.

Looking at the server code in the game it uses:

self.Server = socket.bind("*", self.Port)

This socket.bind() is actually a short of short form for socket.tcp or socket.tcp6 then sock.bind, sock.listen, as mentioned in the LuaSocket manual. It transforms "*" into "0.0.0.0" and then does a getaddrinfo() and steps through the results. As soon as it gets a result, it returns the newly formed listening tcp server socket from this single entry.

You can view socket.bind() in socket.lua from the LuaSocket library - it's not in the C part of the library, but in the Lua part.

I see two problems here. It's not clear to me that "0.0.0.0" will return IPv6 results - in fact, I can see that the Mac isn't, but maybe other systems are, or more likely, they bind to both from a single entry. Secondly if there are multiple interfaces on the server, maybe it's possible for getaddinfo to return multiple results, leading us to only forming an server on some of the results? I haven't checked this second item out yet.

Anyway, to me it looks like this operation has changed on the Mac, because our code has been stable for years, and it definitely worked up till relatively recently.

 

My work around is to iterate around a list, something like this:

local server_addresses = { "0.0.0.0", "::"} -- previously "*"

function GulpNetworking:enableServer(ServerPort)

    local success = false

    -- Bind the host and port
    for _, server_address in ipairs(server_addresses) do
        
        local server, err = socket.bind(server_address, ServerPort)

        if (server) then
            table.insert(self.Servers, server)

            copas.addserver(server, ServerReceiveFunction)

    -- etc.,etc...

NOTE, then this code runs on a Raspberry Pi as a test server I get:

> Started server 0.0.0.0 on port 25561
socket.bind error       address already in use  for     ::
Started server(s)

Which probably means that it seems to has bound to both IPv4 and IPv6 ports with one call.

For testing, I added a 'print_table(addrinfo)' into socket.lua and got the following:

 

1:  
  family: inet
  addr: 0.0.0.0
Started server 0.0.0.0 on port 25561
1:  
  family: inet6
  addr: ::
socket.bind error       address already in use  for     ::
Started server(s)

These results are the same as on the Mac. This means it's calling socket.tcp() .. but I haven't debugged this to see what it's creating inside on the Mac and on Linux. Or whether it's a difference in bind. This will require further investigation.

Other Notes

On previous (Python based) server code, I specifically split the IPv4 and IPv6 server receiving code for various reasons, mostly because, if I remember this correctly, Windows XP didn't support mapped addresses at the time (no IPV6_V6ONLY socket option).

I also used HOST = socket.gethostname() to pass into getaddrinfo() on Windows machines, although on Linux I passed in None. I'm not sure how this related to the native C API... probably maps to a NULL pointer, which appears to be supported on Unix/Linux.

Happy networking!

May 2018 News?

In January https://github.com/robzed/Zex2 was put up on GitHub. We haven't got it running yet.

Quite a bit of work on ForlornFox/EggTag/Escape this year. (Different games, same code-set)

Also started looking at LuaZ80 again, after stubbling across another Z80 in Lua project (by a user that watches LuaZ80, actually!)

Not much robot work - a bit.

Updates on Web pages

Added pages for the GoPiGo line follower and Stairlights.

http://robprobin.com/pmwiki.php?n=Main.Stairlights

http://robprobin.com/pmwiki.php?n=Main.GoPiGoLineFollower

Zex from zip to DMG

Zex download has been updated from a zip file to a dmg.

Why?

Well, download applications on MacOS get isolated (translocated) into a separate location in the file-system to avoid nasty things happening. This means that the application can't find the data files that are next to the original.

The same thing happens with unsigned DMG, which meant we ended up signing the DMG, and getting the user to copy it across to the application folder.

For the download go here - http://robprobin.com/pmwiki.php?n=Main.Zex


P.S. Stu did most of the work here ... I just helped test.

More on the ZX Spectrum Recreate Keyboard

It seems Elite Systems has resolved their problems, and keyboard and app support has been available for a while - please see http://sinclair.recreatedzxspectrum.com/

Also some useful information from Philip Kendall, who has just done a small update to the "*unofficial* Recreated ZX Spectrum FAQ" (as mentioned on the ZX Spectrum Facebook Group) and is located http://www.shadowmagic.org.uk/spectrum/recreated-zx-spectrum-faq.html

My previous post was http://zedcode.blogspot.com/2016/07/notes-on-recreated-zx-spectrum.html.

Regards,
Rob

P.S. At the time of writing (end July 2017) UK retail outlet Game is selling them for £29.99... I'm sure that won't last long.

Live Coding

If you get a chance read this: http://prog21.dadgum.com/221.html

Mentions Bret Victor's interactive stuff - which is WELL worth watching - linked in above article.


We've been using ZeroBrane Studio (https://studio.zerobrane.com/, which supports live programming) but FF has 'grown' beyond interactivity :-(


Did you see that old notch video doing minecraft dev? https://www.youtube.com/watch?v=BES9EKK4Aw4

We should change our code so that it supports this stuff!
 

Notes on The Recreated ZX Spectrum bluetooth Keyboard

Overview

Sadly the tablet and web app have been taken off-line but I knew this before I bought it … It would be wonderful for it to be properly supported under emulators in the game mode. This post list some basic stuff I’ve found out about the keyboard.

Generally Spectrum emulators do work OK with QWERTY mode (Layer B) once unlocked, but really requires support for Game Mode (Layer A).

Unlocking

Unlocking QWERTY mode will be a problem for this keyboard without the App but there is an easy way to run the Javascript to fix this stupid locked mode [7].

Keyboard Modes

The Full User Guide has some information about the two modes - game and QWERTY  [4] - but not enough to really code for it. (But see [6] - although I wonder if it’s accurate - see below).

ZXSP does work with QWERTY mode, but because the two shift keys are overloaded in QWERTY mode, I don’t think this is going to be ideal for Spectrum games. Additionally SYMBOL-SHIFT Q, W, E generate two keys (most of the time) rather than SYMBOL-W key-combos - obviously.

The game mode is interesting - and uses a unique printable character for each of the keys. I guess this is so that it can interact with the web app (and tablet apps) very easily.

The keyboard mapping is pretty simple:

Key    Push+Release
1    ab
2    cd
3    ef
4    gh
5    ij
6    kl
7    mn
8    op
9    qr
0    st
Q    uv
W    wx
E    yz
R    AB
T    CD
Y    EF
U    GH
I    IJ
O    KL
P    MN
A    OP
S    QR
D    ST
F    UV
G    WX
H    YZ
J    01
K    23
L    45
ENTER    67
CAP SHIFT    89
Z    <>
X    -=
C    []
V    ;:
B    ,.
N    /?
M    {}            See note [6]
SYMBOL SHIFT    !$        See Note [6]
BREAK SPACE        %^


So when key 1 is pressed, we get an ‘a’ and when released we get a ‘b’.


Links

[1] Main link: http://sinclair.recreatedzxspectrum.com/

[2] Main support link: http://sinclair.recreatedzxspectrum.com/support.php

[3] Quick Start: http://sinclair.recreatedzxspectrum.com/downloads/recreated_zx_spectrum_user_guide_quick_start_draft_23_i15_v1_0.pdf

[4] Full User Guide: http://sinclair.recreatedzxspectrum.com/downloads/recreated_zx_spectrum_user_guide_l15_v1_0_3.pdf

[5] QWERTY Keyboard shortcuts: http://sinclair.recreatedzxspectrum.com/downloads/recreated_zx_spectrum_keyboard_shortcuts_label_draft_2_g15_v1_0.pdf

[6] There appears to be some conflict here - it might be related to which keyboard you have the Spectrum keyboard selected as - this technical document seems to cover this this … but I didn’t get any difference if I selected ISO or ANSI in my keyboard settings on my Mac http://sinclair.recreatedzxspectrum.com/downloads/recreated_sinclair_zx_spectrum_developer_guide_placeholder_ZX%20Keyboard%20Technical%20Document_a15_v1_1a.pdf

[7] set to Layer A. Then open Web browser and go to: archive.org In the search box go to sinclair.recreatedzxspectrum.com. The code to unlock it is relatively trivial.

Battery Data

This is really cool. My Micromouse Robot - capturing data in the Raspberry Pi gives you a nice chunk of space to store data. In this case capturing battery data over 20 minutes - with a bunch of small runs. Here is a plot of a small amount of this data (400 points out of 10400) - it shows a short run drops the voltage in the 4 LiPo batteries - and the it's restored after the motor load is removed. I'm still running the dsPIC I/O processor, the Raspberry Pi and the Wi-Fi dongle.

Interesting serial port driver hacks

So ... say you've got a Mac. And you've upgraded to El Capitan.

Now your Keyspan serial adapter doesn't work (I have a USA-19HS and older USA-28x) - this used to be a built-in driver, but I guess Apple dropped it. This makes me sad, because Keyspan adapters generally were excellent and didn't cause kernel panics. One obvious option is to install the driver. http://www.tripplite.com/support/USA19HS
 

Say you also have a Prolific-based adapter - not surprising - these are everywhere for a good price. But the last time I installed the drivers I had kernel panics - of course it might be unrelated. You might have to install the native drivers if you want to use a Mac application with it, or see the options below...

Generally, apart from VirtualBox, I never want to install non-Apple drivers/Kext ever, really. (I've considered uninstalling VirtualBox in the past - but currently my machine seems stable even running VirtualBox ... thanks Apple ... famous last words, I know).

But installing drivers is the standard option, I'll admit.

You have the following other choices, as far as I can tell:

1. Use a computer with Yosemite or before. If that sounds like a pain, that's because it is. Especially as your whole family appears to be upgrading.
2. Get an FTDI-based serial adapter off Amazon. Hope it's not a dodgy FTDI clone. This is the route I'd go if it wasn't present season. I *suspect* that the Mac has an FTDI driver built in because it works with things like the Forth board on my the Display Board (which has an FTDI to serial adapter) without installing any drivers...
3. Downgrade to Yosemite.
4. Use a non-Mac ... (NOTE: My Windows PC machine can't sleep without a blue-screen half the time because of serial adapter drivers - none of the ones mentioned here, I should say. Yes I have evidence (a) it started crashing right after installing and (b) it also mentions the specific two drivers in the blue screen. But my work requires them, so hey...).
5. Use a Mac ... but use Ubuntu running in VirtualBox! Ubuntu has Prolific drivers built-in.

VirtualBox has an interesting ability to pass USB traffic to the guest OS and use guest-native drivers, even if the host doesn't have a driver!! So, basically you never need to install a Mac driver (if you can use a Linux app rather than a Mac app).

This link might be helpful for VirtualBox or just normal Linux people ... http://ram.kossboss.com/prolific-rs232/

P.S. To use a Keyspan driver under Linux, you'd need to install it yourself. Debian didn't like the licensing terms, apparently.