How does the command.com shell work with MS-DOS?

Question

On my systems that are running MS-DOS, I change the default command.com to 4DOS for more features.

How was the shell system designed and how did it with the operating system in MS-DOS that it enabled you to swap in a different one?

Answer 1

Traditionally, operating shells are relatively independent of the operating system’s function and the operating system can operate without a shell. Most shells have two modes of operation, interactive mode where they manage the command line and execute commands entered interactively, and batch mode where they run scripts. They’re “standard” programs which handle whatever the input and output system is, and provide their features with no extra special sauce.

In DOS, the “kernel” is provided by two files, typically IO.SYS and MSDOS.SYS (or IBMBIO.COM and IBMDOS.COM). These are loaded at boot-up; they read CONFIG.SYS for configuration, and once all the device drivers have loaded and initialised, they load the shell and start running it. By default the shell is COMMAND.COM on the boot disk, but this can be overridden by the SHELL directive in CONFIG.SYS.

In theory you’d expect the IO and DOS portions of the kernel to provide the “real” operating system, and then COMMAND.COM to follow the standard model of shell design on top of that, but in DOS the distinction between the three parts of the core operating system aren’t as well delineated as you might hope. Because of the way DOS works, with no protection between processes, and using fairly well-documented (or reverse-engineered) functions via the 0x21 interrupt, operating system functions could be implemented in various places, and extended by any number of programs, so the “kernel”, command interpreter and extension TSRs ended up constituting a closely-tied ecosystem... Over the course of its history, DOS ended up relying on COMMAND.COM for certain features, and some of the features DOS provided to COMMAND.COM ended up being extended in ways users expected to find; replacement DOS shells had to provide the former, and ideally respect the latter.

Some examples will hopefully help understand the kinds of issues involved better.

• COMMAND.COM uses a DOS function, 0x210A (“Buffered Keyboard Input”), to read command lines; as standard this provides basic command-line editing capabilities (move the cursor around, insert/delete handling, and F3 command recall), but many TSRs were available to extend this: CMDEDIT, DOS 5's own DOSKEY... As an added subtlety, starting with DOS 3.3, function 0x210A indirectly checked the passage of time to handle midnight — imagine you left a DOS computer sitting at the command prompt overnight; whose job is it to notice that the date has to change? Obviously it’s the operating system’s responsibility, but in single-tasking DOS this required the co-operation of whatever software was running. DOSKEY hooked 0x210A to provide its enhanced command-line handling (adding history), and re-implemented most of the time-related features — but it does not handle midnight, so a DOS 5 computer running DOSKEY can lose a day if it’s sitting at the command prompt at midnight.
• COMMAND.COM allows other programs to run its internal commands (in DOS, even commands such as DIR and COPY are implemented in the command processor); replacement shells ideally need to implement this too. Some third-party programs used this to call internal commands or even add new internal commands to DOS, so replacement shells need to support the same mechanism (interrupt 0x2E — invoking this allows internal commands to be run from outside the shell, and hooking this allows internal commands to be added — and the installable command functions 0x2FAE¹).
• COMMAND.COM installs a CtrlC/CtrlBreak handler and a critical error handler, and replacement shells should do this too.
• COMMAND.COM splits itself into multiple sections: one (small part) for the primary shell, which is always resident (and accessed via function 0x2F55), one containing initialisation code, and finally the so-called transient part which implements most of the functionality. The transient part isn’t needed during execution of other programs (unless they call interrupt 0x2E...), so to provide more memory for applications, it is designed so that it can remain in memory opportunistically without using any allocated memory. The way this works is that it is relocated to the top of memory (with hooks so the other parts of COMMAND.COM can find it), with some sort of signature. Programs which run without using much memory won’t overwrite any of this, and when COMMAND.COM regains control, it checks its transient portion and can then just reuse it without reloading anything. Programs which use enough memory will overwrite the transient portion; when COMMAND.COM regains control, it notices this and reloads its transient portion from disk (which may involve swapping floppies; the COMSPEC variable was used to point COMMAND.COM to the right file).
• In MS-DOS 1.x, program loading is implemented in COMMAND.COM, so anything related to that also lives in COMMAND.COM — for example, interrupt 0x27 (“Terminate and Stay Resident”)!

Most of this was documented at the time, and what wasn’t was quickly discovered. Nowadays you can read the source code for FreeCOM or 4DOS, or even MS-DOS itself, if you’re curious to see exactly how a DOS shell works...

---

¹ See this PC Magazine Tutor article, “Replacing internal DOS commands”, for details.