Sunday, November 11, 2018

Code review: simulate typing

This is part of the Code Review series, even though it's not strictly about FreeDOS.

When we released the FreeDOS 1.2 distribution, I recorded a short video about how to install FreeDOS. I prefer to run FreeDOS on my Linux laptop using the QEMU PC emulator, so that's what I used for my "how to" video. But the thing about QEMU is you don't launch QEMU from a GUI control panel, like you might for other PC emulators like VirtualBox. Instead, you create QEMU's virtual disk and define the QEMU virtual machine by typing commands at the command line. So for my video, I needed to type commands as I talked about what I was doing.

As I tried to record my video, I kept running into problems. I’m just not the kind of person who can type commands at a keyboard and talk about it at the same time. I quickly realized I needed a way to simulate typing at the keyboard, so I could create a “canned” demonstration that I could narrate in my video.

After some searching, I didn’t see a command on my Linux distribution that would simulate typing. I wasn’t surprised; that’s not a common thing people need to do. Instead, I wrote my own program to do it. Here's how I did that.

Overview

Writing a program to simulate typing isn’t as difficult as it may first seem. I needed my program to act like the echo command, where it displayed output given as command-line parameters. I added command-line options so I could set a delay between the program “typing” each letter, with an additional delay for spaces and newlines. The program basically did this:

For each character in a given string:
  1. Insert a delay
  2. Print the character
First, you need a way to simulate a delay in typing, such as someone typing slowly, or pausing before typing the next word or pressing Enter. The C function to create a delay is usleep(useconds_t usec). Use usleep() with the number of microseconds you want your program to pause. So if you want to wait one second, you would use usleep(1000000).

Microseconds means too many zeroes for me to type, so I wrote a simple wrapper called msleep(int millisec) that does the same thing in milliseconds:
int
msleep (int millisec)
{
  useconds_t usec;
  int ret;

  /* wrapper to usleep() but values in milliseconds instead */

  usec = (useconds_t) millisec * 1000;
  ret = usleep (usec);
  return (ret);
}
Next, you need to push characters to the screen after each delay. Normally, you can use putchar(int char) to send a single character to standard output (such as the screen). But you won’t actually see the output until you send a newline. To get around this, you need to flush the output buffer manually. The C function fflush(FILE *stream) will flush an output stream for you. If you put a delay() before each fflush(), it will appear that someone is pausing slightly between typing each character.

Program

Here’s a simple function I wrote to simulate typing. The echodelay() function takes parameters that describe the delay before printing characters, spaces, and newlines. The last parameter is the string to print. The function loops through the string and pauses before printing each character, then flushes the output buffer. The effect is each character seems to appear one at a time, as though someone were typing at a keyboard:
void
echodelay (int chdelay, int spdelay, int nldelay, char *string)
{
  int pos = 0;

  /* add a delay between printing each character in the string,
     depending on the character */

  do
    {
      switch (string[pos])
        {
        case '\0':             /* new line */
          msleep (nldelay);
          break;
        case ' ':              /* space */
          msleep (spdelay);
          break;
        default:               /* character */
          msleep (chdelay);
          break;
        }

      putchar (string[pos]);
      fflush (stdout);
    }
  while (string[pos++] != '\0');
}
With that, it’s a simple process to write a program to parse the command line, set the different delays, and call the echodelay() function to generate the output with the appropriate delays.
/* echodelay.c */

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

void echodelay (int chdelay, int spdelay, int nldelay, char *string);
int msleep (int millisec);
int atoipos (char *string);

int
main (int argc, char **argv)
{
  int opt;
  int chdelay = 0, spdelay = 0, nldelay = 0;

  /* parse command line */

  while ((opt = getopt (argc, argv, "c:s:n:")) != -1)
    {
      switch (opt)
        {
        case 'c':              /* -c nnn */
          chdelay = atoipos (optarg);
          break;
        case 's':              /* -s nnn */
          spdelay = atoipos (optarg);
          break;
        case 'n':              /* -n nnn */
          nldelay = atoipos (optarg);
          break;
        default:               /* unrecognized option */
          fprintf (stderr, "Usage: echodelay [-c millisec] [-s millisec] [-n millisec] [text..]\n");
          exit (1);
          break;
        }
    }

  /* pass all remaining options as text to echodelay() */

  for (opt = optind; opt < argc; opt++)
    {
      echodelay (chdelay, spdelay, nldelay, argv[opt]);
      putchar (' ');
    }

  putchar ('\n');

  exit (0);
}

void
echodelay (int chdelay, int spdelay, int nldelay, char *string)
{

}

int
msleep (int millisec)
{

}

int
atoipos (char *string)
{
  int val;

  /* wrapper to atoi() but always a positive return value */

  val = atoi (string);

  if (val < 0)
    {
      val = 0;
    }

  return (val);
}
And compile it like this:
gcc -Wall -o echodelay echodelay.c
In a shell script, I had commands to print a “prompt,” then simulate typing a command before executing the command. For example, this example to list the contents in your /tmp directory:
#!/bin/sh
echo -n 'prompt$ '
echodelay -c 500 -s 1000 -n 2000 'ls -lh /tmp'
ls -lh /tmp
This is a fairly straightforward C program to simulate typing. I wrote it quickly to do a single job, but it does the job to simulate typing while I narrated my how-to video. In this way, I didn’t need to think about what I was typing while I was trying to describe it. If you need to simulate typing for a similar task, I hope you find this program useful.

Sunday, October 28, 2018

Yes, Digital Mars C/C++ is Boost Licensed

I missed announcing this earlier when it actually happened, so I wanted to put a spotlight on it here: On August 26 2018, Walter Bright of Digital Mars made this announcement:
To answer some questions:

1. Any code (source or binary) distributed as part of the Digital Mars C/C++
development system that is copyrighted by Walter Bright, Digital Mars, or
Symantec, is Boost licensed.

2. Code (source or binary) that is copyrighted by others, such as Microsoft, is
not Boost Licensed. You can download (for free) and use them only as part of the
DMC distribution.

3. Yes, you can still buy the DMC distribution:
https://digitalmars.com/shop.html

Some people prefer to buy (the price is pretty modest) and some people want a
way to remunerate Digital Mars (thank you!), and this is a way to do it.

4. Yes, the DMC compiler is being converted to D!

5. Sorry, the D programming language does not support 16 bit development.

6. Sorry, the Zortech C++ compiler is not Boost licensed, because I was never
able to get permission from all the rights holders. :-(

7. Links:

Digital Mars:
https://digitalmars.com/

Compiler source code:
https://github.com/DigitalMars/Compiler

DMC distribution:
https://github.com/DigitalMars/dmc

Bug reports:
http://bugzilla.digitalmars.com/issues/buglist.cgi?quicksearch=.
If you aren't familiar with the Boost license, the full Boost license text from Digital Mars is in their LICENSE file:
All the files in this package that are copyrighted by:

    Walter Bright
    Digital Mars
    Symantec
    SLR Systems

are licensed by the Boost Software License:

http://www.boost.org/users/license.html

Boost Software License - Version 1.0 - August 17th, 2003

Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:

The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
And yes, that's the same text as recognized by the Open Source Initiative:
Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:

The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
And it's the same text as recognized by the Free Software Foundation:
Boost Software License - Version 1.0 - August 17th, 2003

Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:

The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
The Free Software Foundation, via the GNU project, indicates that the Boost license is "a lax, permissive non-copyleft free software license, compatible with the GNU GPL."

It's great to see more programs being released as open source software! You may know that we are working to update the FreeDOS distribution, to release a FreeDOS 1.3 distribution at the start of 2019. And as part of that work, just as in FreeDOS 1.2, I am very interested to ensure FreeDOS 1.3 includes only Free software and open source software. Seeing the Digital Mars C/C++ compiler released under a license that is recognized as both open source software (OSI) and Free software (FSF) is a big deal for FreeDOS 1.3.

Wednesday, October 17, 2018

Code review: parsing the command line

This is a continuation of the Code Review article series.

At its core, DOS is a command line operating system. Sure, many users might only use the command line long enough to launch their favorite DOS application or game. But for many FreeDOS users and developers, the command line is where it's at.

When I first started working on FreeDOS, I wanted to make sure FreeDOS programs could parse the command line easily and consistently, so every FreeDOS program used pretty much the same syntax. Or as similar as possible.

Let's do a quick review: on DOS systems, most command line programs and utilities use the slash character (/) to start a command line option. Options can be single-letter or single-character options, or they can be entire words. For example, a standard option to tell the program to display a "Help" page is /?. Depending on the program, you might have other command line options, such as /A (perhaps to indicate "all") or /FORCE (to force an action). And on DOS, the options are usually case-insensitive, so /A and /a would usually be treated the same.

Under Unix and Linux systems, the standard way to parse command line arguments is via the getopt() system function. This function allows the program to parse the command line for options that begin with a hyphen (-) such as -a or -o. You can give an argument to an option using the equal sign (=) such as -f=file.txt, or as a following argument such as -f file.txt. GNU extended getopt() to provide a getopt_long() function, which makes command line options more readable. Long options start with a double dash (--) such as --print-all, and can include a short option alternative such as -a (same as --print-all).

For FreeDOS, I didn't want to re-invent the wheel. Why write a completely new library when I can modify something that already does the job? I modified the GNU getopt library to provide a DOS version of getopt() and getopt_long(). I also did some code cleanup to remove some Unix-y things and make the library more suitable for DOS. You can find version 1.2 of the FreeDOS getopt library at ibiblio under files/devel/libs/getopt.

The Readme provides a quick overview and changes from the GNU version, including:

The getopt_long() function works like getopt() except that it also
accepts long options, started out by a slash char.  Both long and
short options may take arguments, which are set off by equals ('=').

Short options are case sensitive.  Long options are not.  This is a
compromise from the UNIX getopt().

The getopt_long() function returns the option character if the option
was found successfully, ':' if there was a missing argument to one of
the options, '?' for an unknown option character, or EOF for the end
of the option list.

getopt_long_only() returns the option character when a short option
is recognized.  For a long option, they return val if flag is NULL,
and 0 otherwise.  Error and EOF returns are the same as for getopt(),
plus '?' for an ambiguous match or an extraneous parameter.

See the foo.c sample program to see how to use getopt_long.

----------------------------------------------------------------------
CHANGES FROM THE GNU getopt_long() FUNCTION:
----------------------------------------------------------------------

I have not yet implemented all features from GNU getopt_long:

 - flag is not used in longopts.

 - longindex is not yet used.

These should be implemented in a future version of getopt_long.

----------------------------------------------------------------------
OTHER ISSUES FOR THE getopt_long() FUNCTION:
----------------------------------------------------------------------

Options must be separated on the command line.  Combining options is
not allowed.  You must write: "foo /a /v" and not "foo /av".  The
second version would try to match a long option called "/av".

Also, you must write: "foo /a /v" and not "foo /a/v".  The second
version would try to match a long option called "/a/v".

I should note this is not a perfect replacement for standard MS-DOS command line parsing. In classic MS-DOS, command line options always start with a slash character, no matter if a long or short option, and you don't need to separate them with spaces. So an MS-DOS program that used both the /A and /V options would interpret /A/V and /A /V as the same. But for the purposes of providing a standard command line experience, I figured this was a good trade-off.

You use the FreeDOS getopt just like the GNU getopt. Here's a sample program:

/*
  The following example program, adapted from the GNU getopt_long
  manual, illustrates the use of getopt_long() with most of its
  features.
*/

/* This program should compile under Linux and DOS equally well. */

#include <stdio.h>

#ifdef unix
#define _GNU_SOURCE
#include <getopt.h>
#else /* assumes DOS */
#include <stdlib.h>
#include "getopt_l.h"
#endif

int
main (argc, argv)
     int argc;
     char **argv;
{
  int c;
  int option_index = 0;
  static struct option long_options[] =
  {
    {"help", 0, 0, 'h'},
    {"verbose", 0, 0, 'v'},
    {"extra", 1, 0, 'x'},
    {0, 0, 0, 0}
  };

  while ((c = getopt_long (argc, argv, "x:hv",
         long_options, &option_index)) != EOF)
    {
      switch (c)
      {
      case 'x':
        printf ("x -> option = %s\n", optarg);
        break;
      case 'h':
        printf ("print help\n");
        break;
      case 'v':
        printf ("verbose mode = on\n");
        break;
      default:
        printf ("?? getopt returned character %c (optopt=%c)\n", c, optopt);
      }
    }

  if (optind < argc)
    {
      printf ("non-option ARGV-elements: ");
      while (optind < argc)
 printf ("%s ", argv[optind++]);
      printf ("\n");
    }

  exit (0);
}

The key is that you define the long options in a structured list, where each item gives the long option name, whether or not the option takes an argument, and the short option. End the list with zeroes.

Each time you call getopt_long(), the function returns the next option on the command line. It does some work behind the scenes to re-order the command line, in case the user put some command line options after regular arguments. For each option returned, use a switched block to trigger an action, such as setting a flag.

After all command line options are processed, the getopt library sets a variable (optind) to the index in the command line vector (in this case, argv) for the first regular option. In many programs, the regular options are files that the program will act against.

Sunday, September 30, 2018

Practical thoughts on MS-DOS source code

I recently wrote that Microsoft released the source code to MS-DOS v1.25 and v2.0 on Github. This is a huge step for Microsoft, and I congratulate them for releasing these older versions of MS-DOS under the open source MIT license!

This source code release was significant because it resolved an issue from Microsoft's previous attempt to open the source code to older MS-DOS. In 2014, Microsoft released the source code to MS-DOS 1.1 and 2.0 via the Computer History Museum. Unfortunately, the license used in the Museum release was a "look but do not touch" license.

My understanding from lawyers who have explained it to me (I am not a lawyer) is that you can be "tainted" by knowledge of proprietary source code, under US law and under similar laws agreed to by partner countries. So anyone who read or studied the source code to MS-DOS 1.1 or 2.0 as it was previously released via the Computer History Museum license was not allowed to contribute to FreeDOS afterwards. We posted several notices to this effect on the FreeDOS website and elsewhere.

But this source code release of MS-DOS 1.25 and 2.0 uses the MIT License, which is not only a recognized open source software license, but compatible with the GNU GPL. This means the "taint" concern is effectively lifted.

While this is great, there's a practical side to the source code release. Note that these are very old versions of MS-DOS. FreeDOS has already surpassed these versions of MS-DOS in functionality and features. For example, MS-DOS 2.0 was the first version to support directories and redirection. But these versions of MS-DOS did not yet include more advanced features including networking, CDROM support, and '386 support such as EMM386.

It's great to see Microsoft open-source these old versions of MS-DOS, but what will be the practical impact on FreeDOS? I think Tom E. answered this well:
“Frankly, not so much. the relevant facts about MSDOS like internal structures, memory layout aso. have been re-engineered/disassembled, documented and commented by Andrew Schulman, Mike Podanowsky, and MANY others, and merged in an almost complete (and almost correct) documented DOS API by Ralph Brown. thanks to them, and there is close to nothing to be learned by studying old MSDOS sources.”

Eric A. adds a similar comment:
“Well, this is mostly interesting for historical research, MS DOS 1.25 had almost no features and 2.0 also is very far away from running most "normal" DOS software.”

So FreeDOS would not be able to reuse this code for any modern features anyway. But for basic features, such as weird edge cases or specific application compatibility, maybe developers can reference this code to improve FreeDOS.

Set your expectations appropriately. Thanks to Microsoft for releasing this source code under an open source software license (MIT) but don't expect this to have much impact on FreeDOS. We've already advanced well beyond MS-DOS 1.25 and 2.0.

Microsoft open-sources old versions of MS-DOS

Microsoft recently released the source code to MS-DOS v1.25 and v2.0 via a repo on Github. This is a huge step for Microsoft, and I congratulate them for releasing these older versions of MS-DOS under a recognized open source software license!

This source code release uses the MIT License (also called the Expat License). From Microsoft's LICENSE.md file on Github:
[MS-DOS 1.25 & 2.0 Source]
Copyright (c) Microsoft Corporation
All rights reserved.
MIT License
Permission is hereby granted, freeof charge, to any person obtaining a copy of this software and associateddocumentation files (the Software), to deal in the Software withoutrestriction, including without limitation the rights to use, copy, modify,merge, publish, distribute, sublicense, and/or sell copies of the Software, andto permit persons to whom the Software is furnished to do so, subject to thefollowing conditions:

The above copyright notice andthis permission notice shall be included in all copies or substantial portionsof the Software.

THE SOFTWARE IS PROVIDED *AS IS*,WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TOTHE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE ANDNONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLEFOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE ORTHE USE OR OTHER DEALINGS IN THE SOFTWARE.
(typos are from original; copied 9/30/2018)

This is the same as the MIT License recognized by the Open Source Initiative:
Copyright <YEAR> <COPYRIGHT HOLDER>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

And the same as the Expat License recognized by the Free Software Foundation:
Copyright (c) 1998, 1999, 2000 Thai Open Source Software Center Ltd

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

The Free Software Foundation (via GNU) says the Expat License (aka "MIT License") is compatible with the GNU GPL. Specifically, GNU describes the Expat License as:
"This is a lax, permissive non-copyleft free software license, compatible with the GNU GPL. It is sometimes ambiguously referred to as the MIT License."

Also according to GNU, when they say a license is compatible with the GNU GPL, "you can combine code released under the other license [MIT/Expat License] with code released under the GNU GPL in one larger program."

That means this source code release of MS-DOS 1.25 and 2.0 removes the concern of "taint" that we had with the previous MS-DOS source code, released via the Computer History Museum in March, 2014. Longtime FreeDOS users may recall that Microsoft posted the source code to MS-DOS 1.1 and 2.0 under a "look but do not touch" license that limited what you could do with the source code. Under the Museum license, users were barred from re-using the source code in other projects, or using concepts from the source code in other projects:
You may use, copy, compile, and create Derivative Works of the software, and run the software and Derivative Works on simulators or hardware solely for non-commercial research, experimentation, and educational purposes. Examples of non-commercial uses are teaching, academic research, public demonstrations, and personal experimentation. “Derivative Works” means modifications to the software, in source code or object code form, made by you pursuant to this agreement.
  • You may copy and refer to any documentation provided as part of the software.
  • You may not distribute or publish the software or Derivative Works.
  • You may not use or test the software to provide a commercial service unless Microsoft permits you to do so under another agreement.
  • You may publish and present papers or articles on the results of your research, and while distribution of all or substantial portions of the software is not permitted, you may include in any such publication or presentation an excerpt of up to fifty (50) lines of code for illustration purposes.
(emphasis mine)

I am not a lawyer, but even I can see this license does not allow users to re-use the MS-DOS source code, especially in open source software projects like FreeDOS. We saw this as a potential risk to FreeDOS; developers who had viewed the MS-DOS source code might "taint" FreeDOS if they later contributed to FreeDOS. To avoid this taint risk, we posted several announcements on the FreeDOS email lists and on the FreeDOS website, including on our FreeDOS History page, to warn FreeDOS developers that they should not view the MS-DOS source code. Anyone who did view the MS-DOS source code could not contribute to FreeDOS:
"Please note: if you download and study the MS-DOS source code, you should not contribute code to FreeDOS afterwards. We want to avoid any suggestion that FreeDOS has been "tainted" by this proprietary code."

But Microsoft's adoption of the MIT License is a significant change. The new MIT License is compatible with the GNU GPL. Therefore, the risk of taint seems to be removed. Congratulations to Microsoft for releasing MS-DOS 1.25 and 2.0 under an open source license!

Saturday, September 8, 2018

Code review: Using catgets/kitten to support different languages

This Code Review article is a repeat from last year, about how to use the Cats and Kitten libraries to support different spoken languages in your programs.

When you write a new program, you probably don't think about spoken languages other than your own. I am a native English speaker, so when I write a new program, all of my error messages and outputted text is in English. And that works well for the many people who have English as their native language, or who know enough English as a second language to get by. But what about others who don't speak English, or who only know a little English? They can't understand what my programs are saying.

The standard Unix method is with a set of C library functions built around language "catalogs." A catalog is just a file that contains all the error messages and other printed text from a program. In the Unix method, you have a different catalog for every language: English, German, Italian, Spanish, French, and so on.

The FreeDOS Cats library was a stripped-down implementation of the Unix library, using a very simple method. Every time you want to print some text in the user's preferred language, you first look up the message string from the catalog using the catgets() function—so named because it will get a string from a message catalog.

In Unix, you use catgets() this way:

  string = catgets(cat, set, num, "Hello world");

This fetches message string number num from message set set, from language catalog cat. The organization of messages into sets allows developers to group status messages into one set (say, set 1), error messages into another set (such as set 7), and so on.

Before calling catgets(), you need to open the appropriate language catalog with a previous call to catopen(). Typically, you have one catalog per language, so you have a different language file for English, another for Spanish, etc. Before your program exits, you close any message catalogs with calls to catclose().

If the string doesn't exist in the message catalog, catgets() returns a default string that you passed to it; in this case, the default string was "Hello world."

I implemented a simplified version of these functions in a FreeDOS Internationalization library called Cats. To save on memory, Cats supported only one open catalog at a time. If you tried to open a second message catalog, the call to catopen() would return an error (-1).

Message catalogs were very simple under Cats. Implemented as plain text files, Cats loaded the entire message catalog into memory at run-time. In this way, you didn't need to recompile the program just to support other languages; you just added another message catalog file for the new language. An English message catalog for a simple program might look like this:

  1.1:Hello world
  7.4:Failure writing to drive A:

The same message catalog in Spanish might look like this:

  1.1:Hola mundo
  7.4:Fallo al escribir en la unidad A:

For example, the string "Failure writing to drive A:" is message number 4 in set 7.

The Cats library was a simple way for developers to add support for different languages in their programs, written in C. And because Cats implemented a Unix standard, it made porting Unix tools to FreeDOS much easier. Once you added the calls to catgets(), all you needed to support other languages was a message catalog that someone translated to a different language. And I kept the Cats message catalogs very simple; they were plain text files.

Cats was a neat innovation, but loading the messages into memory was cumbersome because it used streams. Other FreeDOS developers improved on Cats to optimize the loading of catalogs, reduce memory footprint, and add other enhancements. The new library was noticeably smaller, so we renamed it Kitten.

Because of the optimizations, Kitten used a slightly different API. Since Cats only supported one message catalog at a time anyway, Kitten removed the cat catalog identifier. Once you open a message catalog with kittenopen(), all calls to kittengets() assume that message catalog. You only need to make a single call to kittenclose() before you end the program.

Using Kitten made it much easier to support different spoken languages in FreeDOS programs. Here's a trivial example to put it all together:

  /* test.c */

  #include <stdio.h>
  #include <stdlib.h>
  #include "kitten.h"

  int
  main(void)
  {
    char *s;
 
    kittenopen("test");
 
    s = kittengets(7, 4, "Failure writing to drive A:");
    puts(s);
 
    kittenclose();
    exit(0);
  }

This loads a message catalog "test" into memory, then retrieves message 4 from set 7 into a string pointer s. If message 4 in set 7 isn't found, kittengets() returns the default string "Failure writing to drive A:" into s. The program prints the message to the user, then closes the message catalog before exiting.

Typically, you name the message catalog after the program's name. So the message catalog for the FreeDOS CHOICE program is "choice". Kitten searches for the language file in a few locations on disk, and always appends the value of the %LANG% environment variable, which is typically set to the two-letter language abbreviation: "en" for English or "es" for Spanish. The DOS filename for the English version of the "choice" language catalog is CHOICE.EN, and the Spanish language version is CHOICE.ES.

A limitation to Cats and Kitten is that it can only support single-byte character sets supported by DOS. So Cats and Kitten cannot support Chinese or Japanese, but should do fine with most other languages.
You can find Cats and Kitten at the FreeDOS files archive on ibiblio, under devel/libs/cats/

We made three revisions to Kitten, so the latest version is Kitten revision C, which you can download directly as kitten-c.zip.

FreeDOS contributor Mateusz "Fox" Viste wrote a similar implementation for Pascal programs, called Cubs. You can also find it on ibiblio, under devel/libs/cats/cubs/

FreeDOS developer Eric Auer created a command-line version of Kitten, named Localize, so you can provide internationalization support for DOS Batch (BAT) files. Find it on ibiblio, under devel/libs/cats/localize/

Tuesday, August 21, 2018

New article series: code reviews

I've been thinking of ways to get new developers interested in working on FreeDOS, or at least contributing to FreeDOS in some way. I recognize that DOS is an old concept for some, and as a result, the way you construct programs for DOS can differ to how you'd construct programs for other operating systems such as Linux.

Sure, programs like Choice and Type and Find and other everyday command-line utilities are pretty straightforward. You'd probably write a DOS Type command in the same way you'd write a Unix ‘cat’ command (using streams). But other programs require different methods.

So I thought some kind of “code review” would be helpful, as a way to demonstrate certain DOS programming methods. I'll plan to examine a FreeDOS program and pull apart a specific function or method or programming trick to show how you can use these methods in your own programs.

It will be easiest for me to do a code review on my own programs, and I may start there. But I will try to explore different FreeDOS programs written by different people, so you can see a variety of methods. Every programmer is different, and this will highlight different ways to implement features or optimize execution or minimize footprint when programming in DOS.

Do you have a program you'd like to highlight? I welcome any guest posts in this series. If you have something you'd like to show off, please email me or leave a comment below. I'd be happy to include your article as a guest post here. (If you submit a guest post, it will be easiest if you can share your contribution under the Creative Commons Sharealike [cc-by] public license.)

What programs or methods would you like to see? If you have a suggestion for a “deep dive” into a FreeDOS utility, let me know! My expertise is C, so I am more likely to do code reviews for programs written in C.