What technical reasons prevented the Win9x Virtual Machine Monitor from running multiple threads simultaneously?

Question

Were there any inherent limitations in the way the VMM for Win9x was designed that prevented it from being able to run threads simultaneously if the underlying hardware had multiple cores/processors? Two important aspects stick out to me :

1. Most of the 16-bit Windows code would not be re-entrant.
2. Active DOS applications were always wired to the lower 1MB address space and DOS itself is not re-entrant.

However these do not seem to be inherent limitations in the virtual machine monitor itself. Given that would it theoretically be possible for the primary scheduler in the VMM to execute a thread in parallel that for example does not touch any DOS/Win16 code (how useful such a thread would is a different question entirely given that most of the User32 API thunks down to the 16-bit module).

Answer 1

When using a single core, code which disables interrupts and then performs a sequence of memory operations can guarantee that they will be performed atomically. Additionally, many operations like "inc" would be essentially guaranteed to be atomic in the absence of a page fault. Prior to the introduction of a "compare exchange" instruction on the 80486, this was the most practical way to perform many tasks requiring some kinds of atomic operations. Code which relies upon such techniques, however, won't run reliably in a multi-core system unless code running on different cores can be guaranteed not to act in conflicting fashion upon the same region of memory.

I don't know exactly what constructs Win9x would have executed with interrupts disabled, but any version that could run without "compare exchange" instructions would have had to use such constructs rather heavily.

Answer 2

Theoretically, it might be possible. But practical considerations should not be so easily dismissed.

What @supercat says is true. But part of another reason is already contained in your question: the VMM was not merely a host for applications written against the DOS/BIOS interfaces, it was also itself a client of those same non-reentrant interfaces. This was necessary to implement crucial backwards-compatibility features like the real-mode mapper (the 16-bit I/O driver layer). To be able to run DOS/BIOS code concurrently with other code, you’d need to ensure neither treads on the other’s invariants; I alluded to some of those problems in my answer about a BIOS hard disk driver in Linux. But to save on performance, and to preserve the behaviour of delicate timing loops, the VMM often ran BIOS code in ring 0, giving it full control of the hardware. And some DOS APIs were outright transparently forwarded from the virtual machine to the bare metal.

Of course, this isn’t necessarily insurmountable. As you say, you could simply declare that only a single thread of execution can talk to hardware directly or enter DOS/BIOS code at any given time. In fact, the necessary synchronisation primitive (the critical section) is already there. But that gets you all the performance problems of global locks that we used to know from Linux (the infamous Big Kernel Lock) and still know from CPython (the equally infamous Global Interpreter Lock).

Also, remember that Windows 9x targeted low-end hardware (well, at least lower-end than Windows NT did). Multiprocessing systems were nowhere near as ubiquitous back when those operating systems were being developed as they are today (my barely informed impression is that they remained next to non-existent outside supercomputing throughout the 90s). Implementing SMP support with fine-grained locking takes work, and it might have actually hurt single-threaded workloads (to the point where Windows NT, which did support multiprocessing, required running a separate kernel build in that scenario, just to avoid the synchronisation overhead on single-processor systems). The dearth of synchronisation primitives on early x86 CPUs did not help either. With no market demand for it, there was no reason to expend such effort.