Why did "protected-mode MS-DOS" never happen?

Question

Software written for MS-DOS used DOS extenders as early as on PC/AT (DOS/16M), and starting from i386-based systems, DOS extenders became really widespread.

I'd think it was pretty obvious at the time that the real-mode limitations of both BIOS and MS-DOS had to be left behind sooner rather than later, and DOS extenders (however ingenious technology-wise) were no more than kludgery to bypass those limitations.

I wonder if anybody considered making a "drop-in" replacement for MS-DOS that would be a purely 32-bit protected mode OS? Of course I know about both Windows and OS/2, though I am saying "drop-in" for a reason:

• Both OS/2 and Windows had their own API and programming model that was totally different from the MS-DOS API. So one would have to spend lots of effort to port existing software. Of course it would be impossible to make the API of this imaginary OS fully compatible with the original MS-DOS & DPMI, but at least I guess one could come as close as possible.
• Both OS/2 and Windows did provide MS-DOS virtualization but that was rather a way to run several MS-DOS apps at once; an MS-DOS app would still need a DOS extender if it wanted to access more than 640K of RAM (OK it could use XMS/EMS directly but that was even more kludgery)
• Neither Windows nor OS/2 provided native direct access to video and audio hardware (at least ca. 1992-1994). There were things like WinG but again they brought a totally different programming model compared to how people worked with graphics and audio in MS-DOS.
• Not so sure about OS/2, but the Windows 3.11 32-bit disk access story was less than ideal (very picky about HDD controller hardware and geometry)

So, what exactly made the idea of such an OS infeasible?

Technology-wise, would 32-bit hardware drivers be the biggest problem? Because, kernel-wise, this imaginary OS could have been built on top of the already existing DOS extender technology, just replacing real-mode 16-bit disk access with a 32-bit protected mode counterpart, and providing some abstraction layer over lower memory allocation.

And, marketing-wise, did everyone want multitasking and GUI so much that the idea of a single-tasked OS was no longer appealing by definition?

Answer 1

16-bit protected mode DOS did sort of happen: Concurrent DOS 286 and FlexOS 286 were able to run some DOS applications in protected mode (this involved complex LOADALL shenanigans and revealed bugs in a number of 286 steppings).

What made this infeasible in practice is DOS’s incomplete API. Applications running on top of DOS generally had to perform a number of hardware-specific applications, in particular segment arithmetic for memory management, direct video memory access, and for any hardware not encapsulated by a DOS API, direct hardware access. Given that they had to have some level of direct hardware access in any case, many applications used more than they strictly had to, compounding the problem.

As a result, it wasn’t possible to shift most DOS applications into protected mode without rewriting them anyway, and ideally expanding DOS to provide a much bigger API.

For example, in more detail, 16-bit real-mode programs can use any value they want in segment registers, and DOS programs commonly access memory by changing segment registers. In protected mode, segment register values are selectors, not parts of real memory addresses, and those selectors have to be set up; an operating system trying to run an untranslated DOS program in protected mode would have to catch accesses to unmapped selectors and set them up on the fly (but the amount of selectors is limited).

On top of that, DOS applications were written as 16-bit programs; extending them to run in a 32-bit mode would have required rewriting them, or at least, recompiling them so that they would use 32-bit registers instead of 16-bit registers and actually benefit from the extra address space (and anyone who’s done any amount of 16- to 32-bit porting would tell you that there tend to be many hidden assumptions which break when doing this).

DOS applications written using a DOS extender weren’t DOS applications; they were applications written to the DOS extender’s API. So even the transition to DOS-extended applications is really a transition to a new operating environment, and the DOS extender’s job was to translate that environment to run on top of DOS.

By 1992-1993, there wasn’t much appetite for a new DOS-like operating system in any case — Windows 3.x was the new platform of choice for application development at least. DOS lived on for a few more years, but mostly for games.

Because, kernel-wise, this imaginary OS could have been built on top of the already existing DOS extender technology, just replacing real-mode 16bit disk access with a 32bit protected mode counterpart, and providing some abstraction layer over lower memory allocation.

In effect, that’s what the Windows and OS/2 DOS boxes provide, minus the abstraction layer, and still using real mode (or rather, V86 mode) for DOS applications. Without rewriting the applications running inside them, the benefits are systemic rather than per-application: the ability to multi-task, the ability to use more memory than is actually available; pretty much the only benefit for DOS applications is slightly more conventional memory in an OS/2 DOS box.

And, marketing-wise, did everyone want multitasking and GUI so much that an idea of a single-tasked OS was no longer appealing by definition?

Without rewriting DOS applications, there wasn’t really any other benefit (see above).

Answer 2

And, marketing-wise, did everyone want multitasking and GUI so much that an idea of a single-tasked OS was no longer appealing by definition?

In a word, yes.

The 80386 came to market in 86/87, well after the Macintosh (and Windows) had turned the GUI into something people really wanted to have. Even in the DOS space, many apps were developing into something GUI-like, using custom text or graphics based UI frameworks.

Beyond that, multitasking was becoming more important too. The closest DOS had to multitasking was in the form of TSR's, named after the Terminate and Stay Resident API, that enabled them to remain in memory after the process terminated. A TSR could hook into a few interrupts, listen to keypresses, and pop up a utility like a notepad or calendar when you pressed a hotkey. This worked, but very imperfectly in comparison to the alternatives already available in other platforms. (MacOS Desk Accessories, Windows multitasking, MultiFinder, etc.)

The result of this is that by the time the 80386 rolled around, system software was already obviously deficient in many more ways than would be addressed by a text-mode single-tasking DOS-like OS. There just wouldn't have been the market for it, outside of the hobbyist space. (And keep in mind that 386's in this timeframe were easily in the $10-20K range in today's dollars. They weren't mass market hobbyist machines.)

Note, though, that there were many successful uses of 80386-specific functionality from the later 80's. They just didn't necessarily take the form of a specific new OS.

A few examples:

• An 80386 CPU had enough capability to provide EMS support in software. If you had a DOS program with EMS support, a 80386 could provide access to extra memory that way. (Without an EMS expansion board like in previous CPU's)
• DesqView/386 and Windows/386 let you use the 386 to multitask your existing DOS software.
• OS's like PC-MOS/386 would let multiple users share a single 386 machine over terminals and run DOS software over the terminal link.

These aren't a bespoke 386-specific OS, but they were useful to people and they generally had the benefit of working with software they already had.

Answer 3

The programs that you already had and needed to run were not protected mode programs, they needed to be run in 16-bit real mode anyway.

DOS drivers used BIOS for I/O, which also was 16-bit real mode API.

If you simply change DOS to be 32-bit protected mode OS, you need to provide a 16-bit real mode for programs. And for running BIOS code. A lot of switching back and forth between protected and real modes.

The point is if you already have a BIOS interface to access say a SCSI hard drive, through BIOS ROM onboard the card, then you need a protected mode driver for that. Or just run in 16-bit real mode and use the disk adapter BIOS for access.

Many programs bypassed DOS and used the BIOS directly, for various things, be it a video mode change or something similar. So you would need to have an interface anyway to provide access to 16-bit real mode BIOS, and in the case of SCSI disk, if you don't use the original BIOS interface, your driver needs to then provide emulation for the disk BIOS.

If you run EMM386 or similar memory manager, then actually EMM386 has already set the CPU into protected mode for memory mapping purposes, and it has also created a virtual 86 mode task where the DOS then continues to run like in real mode and so DOS, BIOS access and programs can continue to run in real mode. The EMM386 mainly handles the memory mapping to provide rest of the memory as EMM to the virtual 86 mode task.

If you don't use EMM386 or similar manager, but only use HIMEM or similar memory manager to provide XMS, it will also switch the CPU to protected mode momentarily to allow for copying conventional memory to XMS or back, but the CPU will be running in actual real mode, not in virtual 86 mode.

If a program such as a game wanted to run in protected mode, it can still be run in protected mode, and use DOS and BIOS through real mode callbacks of the protected mode extender library the game uses.

As the system worked, there was no need to write a DOS which would run in 32-bit protected mode.

Additional advantage of running in virtual 86 mode is that it enables hardware emulation with a TSR. If you had a Gravis UltraSound, you could simply install a TSR which asked the EMM386 to trap IO port accessess and call the emulator when these ports were accessed. It made emulation of Sound Blaster PCM DAC, Adlib FM synthesis and Roland MPU MIDI interface possible.

So if you wanted to run any program under virtual 86 mode, and under protected mode DOS, you would need to also emulates BIOS interface. And yet allow programs to use the hardware like display adapter or sound card as they wished which bypasses protection, unless you provided an API for that which programs must use - which starts to sound like Windows already.

Additionally, if a program wanted to set up the protected mode itself, it can't be run if the CPU is already in protected mode and the program is started in the virtual 86 mode task, so it would not be compatible. Some demos and games required this and the PC needed to be started without EMM386 driver for them to work.

So a protected mode DOS would not have been a real DOS, it would have been a new OS, requiring the write or porting of new programs, or requiring the write of DOS compatibility emulation layer for running existing DOS programs, and it would have had many places for problems and incompatibilities running old DOS programs.

Edit: I just saw a short description of why FreeDOS is not 32-bit, and it says the same things I mentioned, i.e. "What would a 32-bit DOS look like" anyway since it does not exist. It would not be a DOS. And 32-bit version of FreeDOS kernel does exist, but it is meant for embedded systems which need custom applications that don't exist and need to be written anyway, so it has no use for running existing real mode DOS apps.

Answer 4

There was at least one proposal for a 32-bit OS that would have been a drop-in replacement for MSDOS: Marc Perkel's NovOS, submitted to Novell in 1991. The intention was that existing MS-DOS programs would run as normal, NovOS-aware 16-bit programs could access 1Mb of linear memory rather than 640k, and NovOS-aware 32-bit programs would have a 32-bit 4G address space.

In practice Novell decided to buy DR-DOS instead. This had some protected-mode features such as DPMS (which allowed protected-mode drivers) but did not implement the range of features listed in the NovOS proposal.

Answer 5

There were several attempts to introduce an ABI like this, including VCPI (Virtual Control Program Interface) and DPMS (DOS Protected-Mode Services), but the one that came closest to being a standard was DPMI (the DOS Protected-Mode Interface) 0.9. It originally was shipped with Windows 3.0.

However, this was not available in MS-DOS without loading Windows—at least for PC-compatibles. The Japanese version of MS-DOS for the PC-98 did support it, as did DR-DOS. But because it wasn’t available on most machines, nearly all software that used it loaded their own “DOS extender” instead, such as the DOS4G/W runtime distributed with the Watcom compilers. Gamers would argue at the time about whether Doom really ran on DOS, or was its own OS. Some extenders went even further, like the Voodoo Memory Manager of Ultima VII that used an undocumented feature of the 80386 processor to enable 32-bit memory access on DOS.

By the time IBM and Microsoft were considering a protected-mode operating system, they already saw Windows or OS/2 as the future, so they abandoned the idea of a protected-mode DOS.

The DOSBox emulator at one point implemented the DPMI standard, but removed it because all software that used DPMI came with their own DOS extenders anyway—which often did not follow the official spec. We therefore tend to think of these not as programs running on the DOS Protected-Mode Interface, but as DOS programs that supply their own 32-bit kernel.

Answer 6

In a nutshell: because at that time everybody wanted GUI and multitasking.

In the other answers to this question, clever people have listed plenty of technical reasons why it would have been very problematic and troublesome to do so. It makes a good reading.

However, the real reason was the frenzy for shiny graphical interfaces where you could resize your windows and the programs running in them. (Not that anybody really had used that feature for anything useful, but that's another topic.) PC technology was (is) very much driven by market needs and trends.

Answer 7

There was multitasking DOS called Wendin DOS constructed with their Wendin Operating System Toolkit (OST.) The toolkit internally appeared to me to be inspired by VMS. To get around the non-rentrant PC BIOS calls int10, int13 etc they either used mutexes or reimplimented the functionality. The DOS was of course a reimplementation of MSDOS syscall interface. The toolkit was also used to implement Unix and VMS work-alike products.

I do remember being shown a multitasking MSDOS 4.x running on a PC/AT machine about the time MSDOS 3.1 was released. I was told this multitasking 4.x was a version restricted to the Asian market.

Looking back the 286 architecture with its 64kb segments really distorted development. Real mode Windows engaged in impressive gymnastics to emulate the 286 protected mode. The Apple Mac, Atari ST and Amiga had the 68000's flat memory from the start - PCs had to wait until the 386.

Answer 8

I'd think it was pretty obvious at the time that the real-mode limitations of both BIOS and MS-DOS

Real-mode limitations are not of BIOS or MS-DOS. Real-mode uses a 16bit addressing + segmentation, which gives 20bit addressing at total (+HMA), and that allows you to access 1Mb of memory in theory, and 640K (+UMA) on PCs. Given that the amounts of memory quickly grown over 1Mb, that addressing model started to represent the limitation by its inability to access whole memory.

To overcome that limitation, your program needs to be re-compiled for different addressing model. The role of DOS extender in that process is explained below.

and DOS extenders (however ingenious technology-wise) were no more than kludgery to bypass those limitations.

The primary goal of the DOS extenders was to minimize the porting efforts of the DOS software authors, when they wanted to switch to protected-mode addressing from real-mode addressing. Extenders usually were shipped together with the compiler tool-chains (eg Borland C, Watcom C), and were allowing to re-compile a well-written C program for protected mode without any modifications. After such re-compilation, the former 16bit pointers were turning into 32bit ones, so even without a single change, the program could already address much more memory than before.

You could, of course, re-compile for win32, rather than for a DOS extender. The aforementioned compilers offered such option too. But the amount of porting efforts would then be much bigger. So the DOS extenders were just a part of a porting layer, and nothing more. The slightly paradoxical situation happened because there was no DirectX or alike API for graphics on PCs yet, so the best graphics libraries were exactly those ported to prot mode with the DOS extenders. That led to the rapid development of DOS extenders for a short while, as if they were a thing on their own, rather than just a cludgy porting layer. But the development of DOS extenders dropped almost immediately with the appearance of DirectX.

I wonder if anybody considered making a "drop-in" replacement for MS-DOS that would be a purely 32-bit protected mode OS?

Of course. There are at least the following ways of doing so:

1. Make a single-tasking protected mode OS that gives a direct hardware access for the DOS programs. A few examples were already pointed out, let me just add FreeDOS-32 to the list.
2. Make a multi-tasking OS with DOS VM (Windows, OS/2).
3. Make an OS+DOS extender+compiler tool-chain such that you can re-compile your DOS program for that OS with minimal porting efforts. This probablly doesn't classify as a "drop-in" replacement though.

Of course it would be impossible to make the API of this imaginary OS fully compatible with the original MS-DOS & DPMI, but at least I guess one could come as close as possible.

There exist PDOS/386 that implements a "theoretical 32-bit MSDOS API" (citing their site). So maybe this is what you are looking for, but you'd need to port your program to that API in order for it to work there. The VM-based solutions, OTOH, allow you to run the DOS programs as they are, without any porting.

an MS-DOS app would still need a DOS extender if it wanted to access more than 640K of RAM

As was already said, this is a limitation of real-mode addressing model, i.e. the hardware limitation in CPU. You need to re-compile your program, and that may or may not require a DOS extender - depending on how many porting efforts you are going to spend.

So, what exactly made the idea of such an OS infeasible?

If you mean a single-tasking DOS-compatible protected-mode OS, then, despite many examples of such OSes, no one uses them today. Multitasking OSes have both the DOS VM for running unmodified DOS apps concurrently, and the DOS extenders to run the "re-compiled versions" of them. They also offer stability, so that you can make sure no DOS app can hang or harm any other DOS or native process within that OS.

Because, kernel-wise, this imaginary OS could have been built on top of the already existing DOS extender technology, just replacing real-mode 16-bit disk access with a 32-bit protected mode counterpart, and providing some abstraction layer over lower memory allocation.

That's a pretty good description of windows-95 actually. It is built around the DOS extender technology, it uses int21 internally quite a lot, and if you disable some 32bit driver, the calls to it would be thunked to real-mode bios. So you may actually think of windows-95 as a very sophisticated DOS extender, with win32 API implemented on top.

And, marketing-wise, did everyone want multitasking and GUI so much that the idea of a single-tasked OS was no longer appealing by definition?

Given that multitasking OSes provide both DOS VM and DOS extender compatibility (DPMI) - yes. All the possible needs are covered and no one needs a single-tasking OS today. And given your desire to run the DOS programs with the direct hardware access, such OS will also be unstable and insecure, and any DOS app can wipe your entire HDD under such OS.

Maybe you can provide more details on what exactly problem you want to solve. Then perhaps there would be the more concrete pointers to the existing solutions.

Answer 9

When not using a memory manager, real-mode x86 programs can be viewed as having very fast access to 64K of data stored in RAM, and moderately fast access to another ~500K (depending upon program size). Using 80286 protected mode would allow programs to have fast access to 64K of data storage, and slowish access to another ~16,000K. Using real mode along with a memory manager would the allow the same kinds of access to 64K/~500K as ordinary real mode, plus a means of copying loading/storing chunks of storage in ways that have a high per-transaction cost, but low per-byte overhead.

For many tasks, having moderately fast access to ~500K and block-based access to expanded memory is more useful than requiring that anything beyond 64K be accessed using means that was slower than real-mode access.

It's a shame that the people who designed Intel's protected mode never recognized what was useful about the 8086 design: there was no need to the CPU to have any kind of knowledge about individual segments--switching between segments, in both code and hardware, was as simple as loading some 16-bit registers. On the 80286, by contrast, switching segment registers required loading segment descriptors--a much slower and more involved process.