Reader

A while back I was able to free up some space over at the Floppy Museum. Almost 200KB! In #DOS, #286 and #floppy disk terms, that's quite a bit, and I've been scratching my head and soliciting advice for what to with all that space.

A recurring suggestion – and one that resonates well with the retro nature of the museum website – has been to add a guestbook. And as is typical for me, my initial reaction was something along the lines of “how hard can it be?”.

Famous last words.

The problem(s)

Looking at the server specs, hardware and software come together in perfect harmony to thwart any reasonable approach to adding a web-site guestbook. There's no way to execute server-side code through the web server, so it was obvious from the start it could not be done in the traditional way.

If you're truly curious about how the server works, do check out the above links before reading on.

Going old(er) school

As those who lived and played before the World Wide Web took over everything will know, the Internet is for more than flash(y) web sites and point-and-click (or tap) user interfaces. Not only are those user interfaces very often abominations in their own right, web browsers have turned into more complex creatures than the operating systems they run under...but I digress.

Having used a lot of BBSes in my youth (have you seen this awesmoe documentary? It's free!), and more recently spun up a Telnet BBS of my own, a semi-obvious idea came to mind: A Telnet-like guestbook service!

Take 1 – rmenu

I'm not even sure how I came across this tool, but I found it over at VCFed (user wiwa64). This clever little thing listens for telnet-type connections and can be used to view files and run programs on a DOS machine. It includes some not-so-rudimentary attempts at forwarding screen and keyboard activity between client and server, so interactivity is actually possible.

This all worked surprisingly well in DOSBox. I put together a simple menu configuration for rmenu and spent too much time creating a .BAT file that would welcome and interrogate the user, then invoke sed (yes, for DOS) to parse the input and make the resulting HTML somewhat safe and readable:

:HTML
rem This block is used if I want separate files per post
rem Uses NDOS/4DOS function to generate a unique file name
::SET OUTFILE=%@UNIQUE[.]
::SET _TEE=`| tee /A`
::SET _R=`>`
::SET _RR=`>>`

rem This appends to an existing guestbook (_TEE is just an append-redirect)
SET OUTFILE=
SET _TEE=`>>`
SET _R=
SET _RR=

rem Output file/table header
IF NOT X%OUTFILE%==X TYPE guests_h.htm %_R %OUTFILE% 
ECHO `<TR ALIGN="LEFT" VALIGN="TOP"><TD ALIGN="LEFT" VALIGN="TOP">` %_TEE% guests_b.htm %_RR% %OUTFILE%

rem Output previously-created timestamp, then prepare the next table cell
ECHO %_DATE %_TEE% guests_b.htm %_RR% %OUTFILE%
ECHO `</TD><TD ALIGN="LEFT" VALIGN="TOP">` %_TEE% guests_b.htm %_RR% %OUTFILE%

rem Output the name given during the interrogation, then begin the next cell
ECHO %name% %_TEE% guests_b.htm %_RR% %OUTFILE%
ECHO `</TD><TD ALIGN="LEFT" VALIGN="TOP">` %_TEE% guests_b.htm %_RR% %OUTFILE%

rem Feed the input text through sed(1) to remove HTML tags and convert newlines to <br>
sed -e `s/<[^>]*>//g` -e `s/$/<br>/g` TMPFILE.TXT %_TEE% guests_b.htm %_RR% %OUTFILE%

rem Close out the table row and optionally the whole page
ECHO `</TD></TR>` %_TEE% guests_b.htm %_RR% %OUTFILE%
IF NOT X%OUTFILE%==X ECHO `</TABLE></BODY></HTML>` %_RR% %OUTFILE%

rem Clean up, then assemble the final, update guests.htm in the HTML directory
DEL /Q TMPFILE.TXT
IF NOT X%OUTFILE%==X REN /Q %OUTFILE% %@NAME[%OUTFILE%].htm
IF EXIST \FLOPMUSE\NUL TYPE guests_h.htm guests_b.htm guests_f.htm > \FLOPMUSE\guests.htm

GOTO END

Yes, I know, parsing HTML with regular expressions is considered harmful. And here I was thinking that would be the biggest problem .. but alas.

So what went wrong?

The real problem with this approach became clear as soon as I deployed it on the actual museum hardware. Whereas DOSBox usually emulates a 32-bit CPU – and of course because I've only tested in plain DOS, without DesqVIEW – the intended habitat of this contraption is quite different.

A 286 can multitask. Not easily, not in any kind of pretty fashion – and certainly not real-mode 16-bit DOS programs. There are operating systems that run on a 286 and can multitask reasonably well. Some examples are Microsoft/SCO Xenix, Novell Netware 286 and my personal favourite, OS/2 1.x. Common to all of them is that they run entirely in protected mode, and while they all happily multitask native software, neither of them can do that with DOS programs. As far as I know, OS/2 is the only one of them that can even run DOS programs, and it does so one at a time and switches to real mode to do so.

DesqVIEW can't multitask for sh*t on a 286. That is to say .. not without help. In the case of the Floppy Museum, this help comes in the form of the C&% SCAT chipset. It provides hardware-assisted LIM EMS, which lets DesqVIEW map alternative memory regions into the conventional memory area (in the case of this chipset/driver, from 256 to 640KB). Along with clever use of timer interrupts and whatever else it's doing, it can actually multitask .. somewhat.

Enough, indeed, to run an FTP server, an IRC server, whatever this project will lead to, and of course the web server – and looking pretty darned good while doing so.

What it can't do, however, is any meaningful kind of virtualization. Specifically, it's missing what is arguably the biggest and most ground breaking feature of the 386: The virtual 8086 mode. This is a kind of virtual machine mode where DOS programs can run in a virtualized address space, and with all hardware access (I/O ports, etc.) managed by the underlying operating system (which itself is running in protected mode).

How is this relevant for rmenu though? It turns out that the way it manages to pass screen output to the client and keyboard presses to the “server” expects to be all alone on the machine. It will simply return whatever the BIOS has written to the screen, and will only pass input as if it is in sole control of the keyboard buffer.

This doesn't only affect rmenu, but also DesqVIEW itself any any other application it runs. While programs that only use BIOS screen writes can be controlled reasonably well by DesqVIEW in this situation, any program doing direct screen writes will bypass that entirely. With v86 mode, each application would have had its own virtualized screen, so rmenu could have lived in blissful ignorance thinking it was the only program running on the CPU. As it is, however, anything but the memory regions swapped in and out of EMS is effectively the Wild West: Anything written to the screen will trample all over anything that is already there, no matter which program wrote it.

Similarly, anything trying to figure out what is on the screen will get whatever is there, whenever it shows up. And this is where rmenu gets confused: Timing is messed up by the time slicing DesqVIEW is doing, and since it's not the only thing writing to the screen, capturing output from the program it's running is disturbed.

In the end, the experience for the user was absolutely horrendeous: Interactivity was absolutely bonkers, with input and output being horribly delayed and even botched. It worked perfectly in DOSBox, but in meeting the real world^Whardware, it came up short.

Take 2 – an attempt at C, C++, or whatever this is

Realising that attempting to hand screen and keyboard control over to a .BAT file and proxying input and output wasn't going to work, the obvious alternative seemed to write a standalone TCP/IP-enabled program to receive the input for the guest book.

The mTCP package comes with source code and some sample code to help clueless people like myself get started. With some helpful hints from @mbbrutman@mastodon.sdf.org (the author), I soon had a working OpenWatcom 1.9 development environment. Being the loon I am, I wanted to do my work in DOS – I just had to work around some limitations of the DOS environment (like making compiler invocation command lines shorter by moving include- and header file paths closer) first.

Long and unfinished story a lot shorter: I am absolutely useless at programming on this level. I haven't touched C or C++ in many decades, so everything I've done so far is largely copying, pasting, endless amounts of trial and error, and basically behaving like a monkey who doesn't even know it's writing a novel.

I mean .. look at this. I don't even know what I'm looking at myself:

while ( !done ) {
    // Service the connection
    PACKET_PROCESS_SINGLE;
    Arp::driveArp( );
    Tcp::drivePackets( );

    if ( mySocket->isRemoteClosed( ) ) {
        fprintf( stderr, "\nRemote end closed connection\n" );
        done = 1;
    } else if ( recvRc > 0 ) {
        msglen = strlen((char *) recvBuf);

        strncpy(msg, (char *) recvBuf, 1);

        switch ( msg[0] ) {
        case '\n':
            if ( added_cret == 0 ) {
                nlcount++;
                added_cret = 0;
                strncpy(entry+strlen(entry), msg, 1);
                sock_printf("\r\n");
            }
            break;
        case '\r':
            if ( entry[strlen(entry)-1] != '\n' ) {
                strncpy(entry+strlen(entry), "\r\n", 2);
                nlcount++;
                nlcount++;
                added_cret = 1;
            }
            sock_printf("\r\n");
            break;
        case '\b':
            if ( entry[strlen(entry)-1] != '\n' &&
              entry[strlen(entry)-1] != '\n') {
                fprintf (stderr, "Deleted last char\n" );
                entry[strlen(entry)-1] = '\0';
                sock_printf("\b");
            }
            break;
        case '\127':
            if ( entry[strlen(entry)-1] != '\n' &&
              entry[strlen(entry)-1] != '\n') {
                fprintf (stderr, "Deleted last char\n" );
                entry[strlen(entry)-1] = '\0';
                sock_printf("\b");
            }
            break;
        case '#':
            entry[strlen(entry)-nlcount] = '\0';
            recvRc = mySocket->recv( recvBuf, sizeof(recvBuf) );
            done = 1;
            break;
        default:
            nlcount = 0;
            strncpy(entry+strlen(entry), msg, 1);
            sock_printf(msg);
        }
    }
}

At the moment, the basic functions of the guestbook sort of works. But then all the fun begins: speaking the Telnet protocol, handling special keys and characters, backspaces and the various end-of-line character combinations, and working with all the different telnet and netcat-like clients out there. For some more horror, consider this attempt at instructing the Telnet client about the characteristics of my “server”:

  uint8_t wont_echo[3] = {255, 252, 1};
  uint8_t will_echo[3] = {255, 251, 1};
  uint8_t dont_echo[3] = {255, 254, 1};
  uint8_t do_echo[3] = {255, 253, 1};
  uint8_t wont_line[3] = {255, 252, 34};
  uint8_t will_line[3] = {255, 251, 34};
  uint8_t dont_line[3] = {255, 254, 34};
  uint8_t do_line[3] = {255, 253, 34};
  uint8_t do_suppress_goahead[3] = {255, 253, 3};
  uint8_t will_suppress_goahead[3] = {255, 251, 3};
  uint8_t escE[2] = {27, 69};
  uint8_t resp[64] = "\0";
  //sock_printf((char *) dont_echo);
  //mySocket->send( wont_line, sizeof(wont_line) );

  mySocket->send( dont_line, sizeof(will_suppress_goahead) );
  PACKET_PROCESS_SINGLE;
  Arp::driveArp( );
  Tcp::drivePackets( );
  mySocket->recv( recvBuf, sizeof(recvBuf) );

  mySocket->send( dont_line, sizeof(do_suppress_goahead) );
  PACKET_PROCESS_SINGLE;
  Arp::driveArp( );
  Tcp::drivePackets( );
  mySocket->recv( recvBuf, sizeof(recvBuf) );

  mySocket->send( dont_line, sizeof(dont_line) );
  PACKET_PROCESS_SINGLE;
  Arp::driveArp( );
  Tcp::drivePackets( );
  mySocket->recv( recvBuf, sizeof(recvBuf) );

Needless to say, it doesn't (yet) work properly. I guess I'll keep working on this..

Desperate measures

A final possibility would be to run a second webserver on the machine. However, all the DOS webservers I've looked at are either too large or don't work with packet drivers. They'd need a whole different set of NIC drivers and maybe even a resident TCP/IP stack, all of which is likely to go far beyond the meager memory and storage resources at my disposal.

This option is still on the table, for the next time I have too much free time on my hands.