Why did DOS-based Windows require HIMEM.SYS to boot?

Question

My understanding is that all versions of Microsoft Windows that ran on top of DOS — that is, the lineage from Windows 1.0 up to Windows ME, even though the reliance on DOS diminished over time — required HIMEM.SYS, the extended memory manager for DOS, to be loaded in order to be able to start in protected mode (thus, from Windows 3.1 on, to be able to start at all).

Contemporary documentation used to make it very clear that the driver was mandatory, but did not elaborate on why it was so. HIMEM.SYS was supposed to provide XMS memory to real mode DOS programs, but since the protected mode Windows kernel was to take over memory management and access all installed memory directly, relying on a 16-bit DOS driver would seem rather nonsensical, especially considering that XMS itself could not be used to access memory arbitrarily but in pieces that had to be copied back and forth between conventional and extended memory.

So, what was the role of HIMEM.SYS with Windows? If it was used just as an intermediary for the Windows loader (which had to be a real mode DOS program) to be able to stuff the insides of Windows into memory before kernel start, how was the switching over to Windows managing the memory accomplished? Did this require HIMEM.SYS to "know" and cooperate with Windows in some way?

Answer 1

Windows in protected mode requires an XMS driver — HIMEM.SYS is the one provided by Microsoft, but others can be used¹ (and the Windows Me kernel includes its own XMS driver). The role of the XMS driver depends on the version of Windows and the mode it’s run in. The basic requirement is that Windows needs to be able to access extended memory, and it needs to do so without losing anything loaded before it (such as the SMARTDrive disk cache).

At the XMS level, there isn’t much cooperation involved: Windows allocates all the available XMS memory (after asking programs such as SMARTDrive to free some memory up if appropriate), and then handles XMS requests itself. XMS includes the necessary mechanism, a chain of hooks such that the calling address for XMS never changes, but the latest XMS handler ends up being called first. This allows Windows to handle XMS requests for programs which were loaded before it started. A good reference for this is Geoff Chappell’s DOS Internals.

Given that Windows, at least in enhanced mode, includes its own XMS handler, one may wonder why it needs to rely on HIMEM.SYS or another driver at all. I suspect that this is to avoid duplication of effort: HIMEM.SYS needs to deal with hardware specifics, A20 gate handling etc.; the Windows driver doesn’t need to worry about that.

Things get more interesting when more complex memory managers are loaded. Such memory managers can only run Windows in enhanced mode if they implement the Global Expanded Memory Manager Import interface, which allows Windows to retrieve all the necessary runtime information from a pre-existing memory manager.

In some cases more dubious shenanigans are required. For example, Windows 3.0 standard mode refuses to run if another enhanced memory manager is already loaded; to work around this, QEMM386 patches the Windows DOS extender (DOSX.EXE) as it loads!

---

¹ Including Microsoft’s own XMSMMGR.EXE, which is used during Windows setup on systems that don’t already have XMS.