scrolls.anduin.net

Reader

Latest posts

from *bsd

As my first post here, I'll try to describe, and perhaps rationalise, how I prefer to run most of my jails.

Background

In case anyone reading this doesn't already know, FreeBSD is a Unix-like operating system tracing its heritage back to the original Berkeley Software Distribution (BSD) from the late 70s. Fast forward to the early '00s and the release of FreeBSD 4.0, and we got a new toy to play with: Jails.

In many ways, jails were the precursor and at least partially the inspiration for several other implementations of OS-level virtualization, with Solaris and Docker being fairly well-known examples. While the implementations differ significantly, the basic idea is the same: Virtualize OS services and resources, but not the hardware, thus achieveing much more frugal virtualization than full-blown VMs.

Basics

Historically, FreeBSD jails were quite simple in their implementation: Aside from separating process spaces (grouping processes and preventing a jail from seeing processes in other jails or on the host) and a rudimentary IP address delegation mechanism, it was essentially (though perhaps not technically) an extension of the chroot mechanism that was first introduced in BSD as early as 1979 – admittedly in a much different shape than today.

Today, this has changed quite a bit, with perhaps the most interesting change being the introduction of VIMAGE, a framework for virtualizing kernel subsystems, and the VNET facility built using VIMAGE, which provides a separate, virtualized network stack to jails.

Add some other killer FreeBSD- and jail-specific features like ZFS and hierarchical, or nested, jails, the PF firewall, and you have the building blocks for some really interesting things!

The problem with classic jails

Classic jails are nice because the lack of a network stack can be considered a security feature: The processes in the jail, even those running as root, are completely isolated from touching the network stack, which means they cannot do anything useful with ifconfig(8), can't modify routing tables, and are generally strictly limited to the IP address(es) you assign to the jail.

Unfortunately, this comes with some drawbacks: Running ifconfig in the jail will reveal much about the network configuration on the host: It will basically give you the full output for the interface(s) on which the jail has IPs, minus the other IPs that are defined there. Consider the following:

$  ifconfig                                                                                                             
ix0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
options=a538b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,HWSTATS>
        ether ac:1f:6b:62:17:1c
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
ix1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
        ether ac:1f:6b:62:17:1d
        media: Ethernet autoselect
        status: no carrier
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        groups: lo
pflog0: flags=0 metric 0 mtu 33152
        options=0
        groups: pflog
ngeth0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=28<VLAN_MTU,JUMBO_MTU>
        ether 58:9c:fc:10:64:f9
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
bridge0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=10<VLAN_HWTAGGING>
        ether ac:1f:6b:62:17:1c
        inet 10.1.2.3 netmask 0xffffff00 broadcast 10.1.2.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        bridge flags=0<>
        member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                port 15 priority 128 path cost 2000 vlan protocol 802.1q
        member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                port 11 priority 128 path cost 2000 vlan protocol 802.1q
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                port 7 priority 128 path cost 2000 vlan protocol 802.1q
        member: ngeth0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                port 5 priority 128 path cost 20000 vlan protocol 802.1q
        member: ix0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                port 1 priority 128 path cost 2000 vlan protocol 802.1q
        groups: bridge
epair0a: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: jail1
        options=200019<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,RXCSUM_IPV6>
        ether 58:9c:fc:10:e6:88
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair1a: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: jail2
        options=200019<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,RXCSUM_IPV6>
        ether 58:9c:fc:10:98:28
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
epair2a: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: jail3
        options=200019<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,RXCSUM_IPV6>
        ether 58:9c:fc:10:22:09
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active

This ifconfig invocation is from within a classic jail running on the same host as a number of VNET jails. Despite not being able to modify anything, I can still see a lot of detail about the host that, one would think, the jail should not be privy to: All the physical and virtual interfaces on the machine, including lo0 (localhost), the bridge0 interface and all the bridge members, the primary interface ix0 and its twin ix1, the pflog0 interface revealing we've got pf running, ngeth0 showing there's netgraph(4) in use, etc.

And perhaps worst of all: The individual epair interfaces for the VNET jails, including their descriptions which – in a well-assembled system – could give away details about what and who else exists on the host.

Clearly, this is not the prison I was hoping for.

The problem with VNET jails

Quite the opposite of classic jails, VNET jails provide a virtualised network stack. This can be connected to the outside world with a virtual cable, by way of the epair interface type, connecting it to a bridge interface on the host. Or you can assign separate physical interfaces, VLAN interfaces, or practically any other interface type. Either way, the jail controls that interface in practically every way, in addition to the routing table (or tables), packet filtering, and most other aspects of the network stack.

This is obviously very powerful, and provides some features that classic jails are (sorely?) missing:

  • An actual localhost interface! Classic jails don't have this, there are just some code shortcuts in the kernel that allow applications that bind to localhost to .. not fail, but there are several situations where this is far from optimal or even desired.
  • The ability to run stuff like nmap(1) from the jail, since it fiddles with routing tables and creates raw network packets.
  • Firewalling with pf(4) (or possibly others; I have not played with those). This includes basic filtering, NAT, rate limiting, and all the other goodness offered.
  • Several network-related sysctl(8) settings can be tuned on a per-jail basis, like datagram sizes, routing settings, TCP congestion control algorithm, etc.

However: Do you really want all the processes in your jail to be this close to all those foot cannons? A simple misconfiguration, local privilege escalation bug, or simply fat-fingering something yourself can wreck havoc on the jail's network configuration – and there is a definite possibility of inflicting damage on other jails, the host, and even other devices on your network.

The limitations of a classic jail suddenly start looking an awful lot like features, don't they?

A possible solution

Without more ado (there's enough of that above), here's how we've solved this at work: We “wrap” a classic jail inside a VNET jail.

Here's how:

1. Create the jail template

The easiest way is to fetch base.txz from the FreeBSD download servers, and estract it in a suitable directory:

# Create template directory. If you use ZFS, a dataset might be preferable:
mkdir -p /data/jails/template.151

# Fetch and extract
fetch -o - https://download.freebsd.org/ftp/releases/amd64/15.1-RELEASE/base.txz \
    | tar --unlink -xpzf - -C /data/jails/template.151

# Create a symlink for use in `fstab(5)`
ln -s /data/jails/template.151 /data/jails/current

2. Create fstab files

For the parent and child jails, create fstab(5) files to nullfs(4)-mount the parts of the base you need into the jails. An added benefit is that this makes the base OS read-only inside the jails, which is probably a good thing:

# /etc/fstab.testjail.parent
/data/jails/current/etc /data/jails/testjail/etc nullfs ro
/data/jails/current/bin /data/jails/testjail/bin nullfs ro
/data/jails/current/lib /data/jails/testjail/lib nullfs ro
/data/jails/current/sbin /data/jails/testjail/sbin nullfs ro
/data/jails/current/libexec /data/jails/testjail/libexec nullfs ro
/data/jails/current/usr/sbin /data/jails/testjail/usr/sbin nullfs ro

# /etc/fstab.testjail
/data/jails/current/bin /data/jails/testjail/root/bin nullfs ro
/data/jails/current/lib /data/jails/testjail/root/lib nullfs ro
/data/jails/current/usr/share /data/jails/testjail/root/usr/share nullfs ro
/data/jails/current/libexec /data/jails/testjail/root/libexec nullfs ro
/data/jails/current/sbin /data/jails/testjail/root/sbin nullfs ro
/data/jails/current/usr/bin /data/jails/testjail/root/usr/bin nullfs ro
/data/jails/current/usr/include /data/jails/testjail/root/usr/include nullfs ro
/data/jails/current/usr/lib /data/jails/testjail/root/usr/lib nullfs ro
/data/jails/current/usr/libdata /data/jails/testjail/root/usr/libdata nullfs ro
/data/jails/current/usr/libexec /data/jails/testjail/root/usr/libexec nullfs ro
/data/jails/current/usr/sbin /data/jails/testjail/root/usr/sbin nullfs ro

3. Create the directory structures for the jails

This is all greatly simplified, but it should give you a starting point. Note how the child jail lives in a subdirectory (/root) inside the parent jail:

# Create the parent jail directory structure
for dir in \
    root dev etc bin lib sbin libexec \
    usr/sbin /var/log /var/run /var/empty
do
    mkdir -p /data/jails/testjail/$dir
done

# Copy some files into the parent jail; necessary for pf to work
cp /data/jails/template.151/etc/pf.os /data/jails/testjail/etc/
cp /data/jails/template.151/etc/protocols /data/jails/testjail/etc/

# Create the child jail directory structure
for dir in \
    root dev bin lib sbin libexec tmp \
    usr/sbin usr/bin usr/include usr/lib usr/libdata \
    usr/libexec usr/share /usr/local/etc
do
    mkdir -p /data/jails/testjail/root/$dir
done

# Copy basic /etc and /var to the child jail
cp -Rp /data/jails/template.151/etc /data/jails/testjail/root/
cp -Rp /data/jails/template.151/var /data/jails/testjail/root/

Using ZFS instead

The mkdir parts for the child jail could be modified to use ZFS (which is what I'd do). This is a skeleton suggestion, but you'd do well in forming your own strategy for how to approach this with ZFS. Personally I see no point using ZFS datasets for the parent jail, as it is largely ephemeral anyway:

# Create the child jail datasets (zfs 'create -p' acts like 'mkdir -p')
for dir in \
    root usr/local usr/home tmp var etc
do
    zfs create -p data/jails/testjail/root/$dir
done

# Create directories in the datasets
for dir in \
    dev bin lib sbin libexec \
    usr/sbin usr/bin usr/include usr/lib usr/libdata \
    usr/libexec usr/share /usr/local/etc
do
    mkdir -p /data/jails/testjail/root/$dir
done

One useful trick I've learned is to keep /var/log inside the child jail in a ZFS dataset homed outside the child jail's path, and simply set its mountpoint to be inside the jail. In case you need to roll back the dataset(s) in the child jail, you can safely do so without erasing your logs. Security people and auditors may even come to appreciate this:

# Create the dataset; make sure it always has space to write logs even if the
# rest of the datasets have run out of space:
zfs create -p \
    -o reservation=64M \
    -o mountpoint=/data/jails/testjail/root/var/log \
    data/jails/testjail/logs

4. Start the jails

We start by creating the epair interfaces. Assuming there is already a bridge0 to attach the dangling cable^Winterface to:

if_a="$(ifconfig epair create)"
if_b="${if_a%a}b"
ifconfig bridge0 addm $if_a
ifconfig $if_a up

Next, we start the parent jail. Note this is crude; a complete jail.conf entry should take care to unmount filesystems on jail shutdown, avoiding mounting things twice, and various other housekeeping:

# Mount the parent filesystems
mount -a -F /etc/fstab.testjail.parent

# Start the jail - set children.max=1 to allow the child jail
jail -c path=/data/jails/testjail name=testjail children.max=1 vnet persist

# $if_a and $if_b is from the interface create script
ifconfig $if_b vnet testjail
jexec testjail ifconfig lo0 127.0.0.1/8
jexec testjail ifconfig $if_b 10.2.3.4/24
jexec testjail route add default 10.2.3.1

# Mount devfs in the jails
mount -t devfs -oruleset=4 . /data/jails/testjail/dev
mount -t devfs -oruleset=4 . /data/jails/testjail/root/dev

Now that we have the parent jail running, starting the child jail is a matter of

# Mount the child filesystems
mount -a -F /etc/fstab.testjail.parent

# Start the child jail
jexec testjail jail -c ip4=inherit path=/root name=child exec.start="/bin/sh /etc/rc autoboot"

At this point you should be able to jexec testjail.child su - and find yourself in a bare, but functional, jail. Run ifconfig to see that only network interfaces pertaining to testjail are visible, and confirm that you cannot make changes. List the jails with jls; it should show two jails – the parent and the child.

Stopping the jails

Since we're cheating and doing everything from the command line here, stopping the jails takes a few steps:

# Stop the child jail by killing all the processes
jexec testjail.child kill -9 -1

# In case there are lingering TCP sockets preventing the child jail from dying,
# use tcpdrop(8) to clean up:
jexec testjail tcpdrop -a

# Destroy the parent jail
jail -r testjail

# Destroy the epair interfaces (find the name using ifconfig if needed; both
# the 'a' and 'b' of the pair should now be visible on the host:
ifconfig $if_a destroy

# Finally, unmount the filesystems
umount -f /data/jails/testjail/root/dev
umount -f /data/jails/testjail/dev
umount -F /etc/fstab.testjail -a
umount -F /etc/fstab.testjail.parent -a

What now?

All of the above, while lengthy, is hardly elegant. Some additional steps may be desirable, e.g. to enable pf, creating a “black hole” around the child jail, and other niceties. A jail.conf taking care of these steps should also be created, which will require a helper script or two.

Enabling the packet filter

In order to use pf, we need a pf.conf file, and some additional device nodes and optionally the pflog interface in the parent jail.

The pf.conf file really only needs a single line for the purpose of this demonstration. We're adding log here for pflog to have any work to do. Place it in the root of the parent jail, like so:

echo "pass log all" > /data/jails/testjail/pf.conf

Then we need to unhide a couple of device nodes in the parent jail, load the rules, and start pflogd. This should be done after starting the parent jail, but before starting the child jail:

devfs -m /data/jails/testjail/dev rule apply path pf unhide
devfs -m /data/jails/testjail/dev rule apply path bpf unhide
jexec testjail pfctl -F all -vf /pf.conf && pfctl -e

# Starting pflogd, excluding some particularly noisy traffic
jexec testjail pflogd 'not host 224.0.0.18'

At this point, if you plan to keep a configuration like this, you may want to make sure you rotate the pflogd files automatically, by adding this to /usr/local/etc/newsyslog.d/jails.conf (or anywhere else you prefer): /data/jails/testjail/var/log/pflog root:wheel 640 7 * $D0 B /data/jails/nextcloud/var/run/pflogd.pid Ask me how I know. :-P

The Black Hole

One of the more “interesting” possibilities with this approach is that after the child jail has been started, there really is no reason to keep the parent jail around. Of course it needs to exist, but most of the filesystems (i.e binaries and libraries) are really not necessary. While nowhere near an actual black hole (you can still access the child jail's files), it has a non-zero value as an obfuscation.

Following startup of the child jail and all the parent jail services (pflogd, etc.), you can simply unmount all the parent filesystems: umount -F /etc/fstab.testjail.parent

However, a side effect of this is that you'll no longer be able to modify the running configuration (update pf rules, etc.). You can no longer simply jexec into the jail – no libraries or binaries are available in their expected locations.

To get around this, you can use some variation of the following: LD_LIBRARY_PATH=/root/lib jexec testjail /root/libexec/ld-elf.so.1 /root/sbin/pfctl -F all -f /pf.conf This will set LD_LIBRARY_PATH to the child jail's path, and invoke ld-elf.so.1 directly (with the binary as argument), since it doesn't exist in /libexec any longer.

Be aware, however, that doing this will expose you to potentially tainted files in the child jail. An alternate approach would be to mount and unmount the parent filesystems, e.g.

mount -F /etc/fstab.testjail.parent -a
jexec testjail <command>
umount -F /etc/fstab.testjail.parent -a

Then again, since these base OS directories are read-only mounted from the template, this should be a minimal risk. If you're attempting to run programs/libraries that live in read/write-mounted directories in the child jail, however – beware.

Putting it all together (sort of)

What's missing now is of course the jail.conf entry. With this in place, you can use the regular service jail start [<name>] and service jail stop [<name>] commands to manage your jails, and you can have them automatically start on system boot.

Correctly assembling such a configuration is a fairly involved task, but I'll leave you with an example that should help you get started. Good luck!

# /etc/jail.conf
## Some defaults
# The default devfs ruleset - this is a default /etc/defaults/devfs.rules
devfs_ruleset = 4;
# We want the venerable rc(8) to manage the child jails
exec.start = "/bin/sh /etc/rc autoboot";
# Set a high securelevel for the child jails (see security(7))
$child_securelevel = 4;
# Disallow modifying hostnames after creating jails
allow.set_hostname = 0;
# Clean the environment
exec.clean;

## Our test jail starts here
testjail {
    # Set a variable for use later - a run directory for some housekeeping
    $rundir = "/var/run/jails/testjail";

    # Basic settings for the parent jail
    host.hostname = "testjail.PARENT";
    path = "/data/jails/testjail";
    vnet;
    # Setting a high value here, so we can use service jails (see rc.conf(5))
    children.max = 21;

    # The fstab for the parent, will be automatically mounted on startup. Also
    # make sure we can mount filesystems inside the parent
    mount.fstab = "/etc/fstab.bbs";
    allow.mount;
    allow.mount.devfs;
    # Allow inspecting ZFS snapshots from within the child jail (optional)
    zfs.mount_snapshot = 1;
    # In case you want PostgreSQL or something
    allow.sysvipc = 1;
    # Keep a console log for the jail
    exec.consolelog = "/var/log/jail_${name}_console.log";

    # Cleanup before starting the jail (In case a previous attempt failed)
    exec.prepare = "umount -f -F /etc/fstab.testjail -a || :";
    exec.prepare += "umount -f -F /etc/fstab.testjail.parent -a || :";
    exec.prepare += "umount -f ${path}/root/dev || :";
    exec.prepare += "umount -f ${path}/dev || :";

    ### Starting the jail
    ## Pre-start commands
    # Create rundir on host
    exec.prestart = "mkdir -p ${rundir}";
    # Create the epair interface; put the names of each end in files in $rundir
    exec.prestart += "ifconfig epair create > ${rundir}/if_a"
    exec.prestart += "sed 's/a$/b/' < ${rundir}/if_a > ${rundir}/if_b"
    # Mount parent jail filesystems
    exec.prestart += "mount -F /etc/fstab.${name}.parent -a || :";
    # Mount devfs in parent jail
    exec.prestart += "mount -t devfs -oruleset=${devfs_ruleset} . ${path}/dev";
    # Unhide pf and bpf device in parent jail
    exec.prestart += "devfs -m ${path}/dev rule apply path pf unhide";
    exec.prestart += "devfs -m ${path}/dev rule apply path bpf unhide";

    ## Post-create commands (run after jail created, but before exec.start)
    # Mount devfs in the child jail with appropriate ruleset
    exec.created = "mount -t devfs -oruleset=${devfs_ruleset} . ${path}/root/dev";
    # Assign jail-side interface to jail
    exec.created += "ifconfig `cat ${rundir}/if_b` vnet ${name}";
    # Assign IP address to jail interface
    exec.created += "jexec ${name} ifconfig $(cat ${rundir}/if_b) inet 10.2.3.4/24";
    # Add default route
    exec.created += "jexec ${name} route add default 10.2.3.1";

    ## Startup commands
    # Assign IP to lo0-interface
    exec.start = "ifconfig lo0 127.0.0.1/8";
    # Load PF rules for parent jail and enable
    exec.start += "pfctl -F all -vf /pf.conf && pfctl -e";
    # Start pflogd
    exec.start += "/sbin/pflogd 'not host 224.0.0.18'";
    # Wait for ping to jail (interface plumbing); assuming this file is
    # autogenerated and the IP includes the netmask
    exec.start += "ip='10.1.2.3/24' ; ping -n -c 15 -t 20 -o \${ip%/*}";
    # Set sysctls (if any are configured)
    exec.start += "(test -s /sysctl.conf && sysctl -f /sysctl.conf) || :";

    ## Create child jail
    exec.start += "jail -c ip4=inherit path=/root host.hostname=testjail \
                   name=child zfs.mount_snapshot=1 children.max=20 \
                   securelevel=${child_securelevel} \
                   sysvmsg=\"new\" sysvsem=\"new\" sysvshm=\"new\" \
                   exec.start=\"/bin/sh /etc/rc autoboot\"";

    ## Post-start commands
    # Unount parent jail filesystems
    exec.poststart += "umount -F /etc/fstab.${name}.parent -a -f || :";

    ### Stopping the jail
    ## Pre-stop commands
    # Mount parent jail filesystems
    exec.prestop = "mount -F /etc/fstab.${name}.parent -a";

    ## Shutdown commands
    # Shut down child jail cleanly, if possible
    exec.stop = "jexec child /bin/sh /etc/rc.shutdown ; sleep 1";
    # Forcibly remove child jail, just in case
    exec.stop += "jls -d name|chroot /root grep -q '^${name}.child$' && jail -Rd ${name}.child || :";
    # Remove any lingering TCP sockets
    exec.stop += "tcpdrop -a || :";

    ## Post-stop commands
    # Unount parent jail filesystems
    exec.poststop = "umount -F /etc/fstab.${name}.parent -a -f || :";
    # Unmount devfs
    exec.poststop += "umount -f ${path}/root/dev || :";
    # Unount parent jail dev filesystem
    exec.poststop += "umount -f ${path}/dev || :";
    # Finally, destroy jail interfaces
    exec.poststop += "ifconfig `cat ${rundir}/if_b` destroy";

    # Clean up rundir
    exec.poststop += "rm -Rf ${rundir}";
 
Read more...

from skrot

A while back I was able to free up some space over at the Floppy Museum. Almost 200KB! In #DOS, #286 and #floppy disk terms, that's quite a bit, and I've been scratching my head and soliciting advice for what to with all that space.

A recurring suggestion – and one that resonates well with the retro nature of the museum website – has been to add a guestbook. And as is typical for me, my initial reaction was something along the lines of “how hard can it be?”.

Famous last words.

The problem(s)

Looking at the server specs, hardware and software come together in perfect harmony to thwart any reasonable approach to adding a web-site guestbook. There's no way to execute server-side code through the web server, so it was obvious from the start it could not be done in the traditional way.

If you're truly curious about how the server works, do check out the above links before reading on.

Going old(er) school

As those who lived and played before the World Wide Web took over everything will know, the Internet is for more than flash(y) web sites and point-and-click (or tap) user interfaces. Not only are those user interfaces very often abominations in their own right, web browsers have turned into more complex creatures than the operating systems they run under...but I digress.

Having used a lot of BBSes in my youth (have you seen this awesmoe documentary? It's free!), and more recently spun up a Telnet BBS of my own, a semi-obvious idea came to mind: A Telnet-like guestbook service!

Take 1 – rmenu

I'm not even sure how I came across this tool, but I found it over at VCFed (user wiwa64). This clever little thing listens for telnet-type connections and can be used to view files and run programs on a DOS machine. It includes some not-so-rudimentary attempts at forwarding screen and keyboard activity between client and server, so interactivity is actually possible.

This all worked surprisingly well in DOSBox. I put together a simple menu configuration for rmenu and spent too much time creating a .BAT file that would welcome and interrogate the user, then invoke sed (yes, for DOS) to parse the input and make the resulting HTML somewhat safe and readable:

:HTML
rem This block is used if I want separate files per post
rem Uses NDOS/4DOS function to generate a unique file name
::SET OUTFILE=%@UNIQUE[.]
::SET _TEE=`| tee /A`
::SET _R=`>`
::SET _RR=`>>`

rem This appends to an existing guestbook (_TEE is just an append-redirect)
SET OUTFILE=
SET _TEE=`>>`
SET _R=
SET _RR=

rem Output file/table header
IF NOT X%OUTFILE%==X TYPE guests_h.htm %_R %OUTFILE% 
ECHO `<TR ALIGN="LEFT" VALIGN="TOP"><TD ALIGN="LEFT" VALIGN="TOP">` %_TEE% guests_b.htm %_RR% %OUTFILE%

rem Output previously-created timestamp, then prepare the next table cell
ECHO %_DATE %_TEE% guests_b.htm %_RR% %OUTFILE%
ECHO `</TD><TD ALIGN="LEFT" VALIGN="TOP">` %_TEE% guests_b.htm %_RR% %OUTFILE%

rem Output the name given during the interrogation, then begin the next cell
ECHO %name% %_TEE% guests_b.htm %_RR% %OUTFILE%
ECHO `</TD><TD ALIGN="LEFT" VALIGN="TOP">` %_TEE% guests_b.htm %_RR% %OUTFILE%

rem Feed the input text through sed(1) to remove HTML tags and convert newlines to <br>
sed -e `s/<[^>]*>//g` -e `s/$/<br>/g` TMPFILE.TXT %_TEE% guests_b.htm %_RR% %OUTFILE%

rem Close out the table row and optionally the whole page
ECHO `</TD></TR>` %_TEE% guests_b.htm %_RR% %OUTFILE%
IF NOT X%OUTFILE%==X ECHO `</TABLE></BODY></HTML>` %_RR% %OUTFILE%

rem Clean up, then assemble the final, update guests.htm in the HTML directory
DEL /Q TMPFILE.TXT
IF NOT X%OUTFILE%==X REN /Q %OUTFILE% %@NAME[%OUTFILE%].htm
IF EXIST \FLOPMUSE\NUL TYPE guests_h.htm guests_b.htm guests_f.htm > \FLOPMUSE\guests.htm

GOTO END

Yes, I know, parsing HTML with regular expressions is considered harmful. And here I was thinking that would be the biggest problem .. but alas.

So what went wrong?

The real problem with this approach became clear as soon as I deployed it on the actual museum hardware. Whereas DOSBox usually emulates a 32-bit CPU – and of course because I've only tested in plain DOS, without DesqVIEW – the intended habitat of this contraption is quite different.

A 286 can multitask. Not easily, not in any kind of pretty fashion – and certainly not real-mode 16-bit DOS programs. There are operating systems that run on a 286 and can multitask reasonably well. Some examples are Microsoft/SCO Xenix, Novell Netware 286 and my personal favourite, OS/2 1.x. Common to all of them is that they run entirely in protected mode, and while they all happily multitask native software, neither of them can do that with DOS programs. As far as I know, OS/2 is the only one of them that can even run DOS programs, and it does so one at a time and switches to real mode to do so.

DesqVIEW can't multitask for sh*t on a 286. That is to say .. not without help. In the case of the Floppy Museum, this help comes in the form of the C&% SCAT chipset. It provides hardware-assisted LIM EMS, which lets DesqVIEW map alternative memory regions into the conventional memory area (in the case of this chipset/driver, from 256 to 640KB). Along with clever use of timer interrupts and whatever else it's doing, it can actually multitask .. somewhat.

Enough, indeed, to run an FTP server, an IRC server, whatever this project will lead to, and of course the web server – and looking pretty darned good while doing so.

What it can't do, however, is any meaningful kind of virtualization. Specifically, it's missing what is arguably the biggest and most ground breaking feature of the 386: The virtual 8086 mode. This is a kind of virtual machine mode where DOS programs can run in a virtualized address space, and with all hardware access (I/O ports, etc.) managed by the underlying operating system (which itself is running in protected mode).

How is this relevant for rmenu though? It turns out that the way it manages to pass screen output to the client and keyboard presses to the “server” expects to be all alone on the machine. It will simply return whatever the BIOS has written to the screen, and will only pass input as if it is in sole control of the keyboard buffer.

This doesn't only affect rmenu, but also DesqVIEW itself any any other application it runs. While programs that only use BIOS screen writes can be controlled reasonably well by DesqVIEW in this situation, any program doing direct screen writes will bypass that entirely. With v86 mode, each application would have had its own virtualized screen, so rmenu could have lived in blissful ignorance thinking it was the only program running on the CPU. As it is, however, anything but the memory regions swapped in and out of EMS is effectively the Wild West: Anything written to the screen will trample all over anything that is already there, no matter which program wrote it.

Similarly, anything trying to figure out what is on the screen will get whatever is there, whenever it shows up. And this is where rmenu gets confused: Timing is messed up by the time slicing DesqVIEW is doing, and since it's not the only thing writing to the screen, capturing output from the program it's running is disturbed.

In the end, the experience for the user was absolutely horrendeous: Interactivity was absolutely bonkers, with input and output being horribly delayed and even botched. It worked perfectly in DOSBox, but in meeting the real world^Whardware, it came up short.

Take 2 – an attempt at C, C++, or whatever this is

Realising that attempting to hand screen and keyboard control over to a .BAT file and proxying input and output wasn't going to work, the obvious alternative seemed to write a standalone TCP/IP-enabled program to receive the input for the guest book.

The mTCP package comes with source code and some sample code to help clueless people like myself get started. With some helpful hints from @mbbrutman@mastodon.sdf.org (the author), I soon had a working OpenWatcom 1.9 development environment. Being the loon I am, I wanted to do my work in DOS – I just had to work around some limitations of the DOS environment (like making compiler invocation command lines shorter by moving include- and header file paths closer) first.

Long and unfinished story a lot shorter: I am absolutely useless at programming on this level. I haven't touched C or C++ in many decades, so everything I've done so far is largely copying, pasting, endless amounts of trial and error, and basically behaving like a monkey who doesn't even know it's writing a novel.

I mean .. look at this. I don't even know what I'm looking at myself:

while ( !done ) {
    // Service the connection
    PACKET_PROCESS_SINGLE;
    Arp::driveArp( );
    Tcp::drivePackets( );

    if ( mySocket->isRemoteClosed( ) ) {
        fprintf( stderr, "\nRemote end closed connection\n" );
        done = 1;
    } else if ( recvRc > 0 ) {
        msglen = strlen((char *) recvBuf);

        strncpy(msg, (char *) recvBuf, 1);

        switch ( msg[0] ) {
        case '\n':
            if ( added_cret == 0 ) {
                nlcount++;
                added_cret = 0;
                strncpy(entry+strlen(entry), msg, 1);
                sock_printf("\r\n");
            }
            break;
        case '\r':
            if ( entry[strlen(entry)-1] != '\n' ) {
                strncpy(entry+strlen(entry), "\r\n", 2);
                nlcount++;
                nlcount++;
                added_cret = 1;
            }
            sock_printf("\r\n");
            break;
        case '\b':
            if ( entry[strlen(entry)-1] != '\n' &&
              entry[strlen(entry)-1] != '\n') {
                fprintf (stderr, "Deleted last char\n" );
                entry[strlen(entry)-1] = '\0';
                sock_printf("\b");
            }
            break;
        case '\127':
            if ( entry[strlen(entry)-1] != '\n' &&
              entry[strlen(entry)-1] != '\n') {
                fprintf (stderr, "Deleted last char\n" );
                entry[strlen(entry)-1] = '\0';
                sock_printf("\b");
            }
            break;
        case '#':
            entry[strlen(entry)-nlcount] = '\0';
            recvRc = mySocket->recv( recvBuf, sizeof(recvBuf) );
            done = 1;
            break;
        default:
            nlcount = 0;
            strncpy(entry+strlen(entry), msg, 1);
            sock_printf(msg);
        }
    }
}

At the moment, the basic functions of the guestbook sort of works. But then all the fun begins: speaking the Telnet protocol, handling special keys and characters, backspaces and the various end-of-line character combinations, and working with all the different telnet and netcat-like clients out there. For some more horror, consider this attempt at instructing the Telnet client about the characteristics of my “server”:

  uint8_t wont_echo[3] = {255, 252, 1};
  uint8_t will_echo[3] = {255, 251, 1};
  uint8_t dont_echo[3] = {255, 254, 1};
  uint8_t do_echo[3] = {255, 253, 1};
  uint8_t wont_line[3] = {255, 252, 34};
  uint8_t will_line[3] = {255, 251, 34};
  uint8_t dont_line[3] = {255, 254, 34};
  uint8_t do_line[3] = {255, 253, 34};
  uint8_t do_suppress_goahead[3] = {255, 253, 3};
  uint8_t will_suppress_goahead[3] = {255, 251, 3};
  uint8_t escE[2] = {27, 69};
  uint8_t resp[64] = "\0";
  //sock_printf((char *) dont_echo);
  //mySocket->send( wont_line, sizeof(wont_line) );

  mySocket->send( dont_line, sizeof(will_suppress_goahead) );
  PACKET_PROCESS_SINGLE;
  Arp::driveArp( );
  Tcp::drivePackets( );
  mySocket->recv( recvBuf, sizeof(recvBuf) );

  mySocket->send( dont_line, sizeof(do_suppress_goahead) );
  PACKET_PROCESS_SINGLE;
  Arp::driveArp( );
  Tcp::drivePackets( );
  mySocket->recv( recvBuf, sizeof(recvBuf) );

  mySocket->send( dont_line, sizeof(dont_line) );
  PACKET_PROCESS_SINGLE;
  Arp::driveArp( );
  Tcp::drivePackets( );
  mySocket->recv( recvBuf, sizeof(recvBuf) );

Needless to say, it doesn't (yet) work properly. I guess I'll keep working on this..

Desperate measures

A final possibility would be to run a second webserver on the machine. However, all the DOS webservers I've looked at are either too large or don't work with packet drivers. They'd need a whole different set of NIC drivers and maybe even a resident TCP/IP stack, all of which is likely to go far beyond the meager memory and storage resources at my disposal.

This option is still on the table, for the next time I have too much free time on my hands.

 
Read more...

from skrot

It used to matter to me that the big fish did not eat all the small fish. This is still true.

It used to matter to me that technology won out because it was better, more fit for purpose, and/or enabled, invited and inspired new ways to think. This is still true.

It used to matter to me that people made informed choices for themselves, and were allowed to do so by their peers. This is still true.

None of this came to pass.

 
Read more...