According to the Oracle VirtualBox 6.0 manual, "Certain rare guest operating systems like OS/2 make use of very esoteric processor instructions that are not supported with our software virtualization. For virtual machines that are configured to contain such an operating system, hardware virtualization is enabled automatically." Any idea which "very esoteric processor instructions" would be meant by this?
1 Answer
As far as I’m aware the difficulty in virtualising OS/2 isn’t due to esoteric processor instructions, but rather esoteric processor features. Specifically, OS/2 uses all the protected mode features available on 286 (OS/2 1.x) and 386 (OS/2 2.0 and later) PCs: segment limits, paging, protection rings... The latter has commonly been given as the main difficulty in supporting OS/2: while most protected-mode PC operating systems use only rings 0 (kernel) and 3 (user-mode), OS/2 also uses ring 2 for privileged code which couldn’t touch the kernel but could access some pieces of hardware (for printers and displays).
Because support for x86-style rings other than 0 and 3 isn’t needed for many operating systems, it probably isn’t a priority for developers of virtualisation tools; bear in mind that most virtualisation tools are developed mostly by companies, and their goal is typically to support specific workloads rather than provide a complete emulation of the original hardware platform. By skipping some support for unused rings, they only lost support for operating systems such as OS/2, and some features of DR DOS (DPMS specifically). On top of that, at least some virtualisation tools rely on protection rings themselves to virtualise the operating system they’re running: if they can’t use hardware-assisted virtualisation, they’ll run their virtual machine manager in ring 0, and run the guest operating system in ring 1 (instead of ring 0). Doing that doesn’t leave much room for the guest operating system to do anything clever with protection rings itself.
This probably isn’t the only feature which causes issues. OS/2 runs in both 16-bit and 32-bit protected mode simultaneously, with frequent switches from one to the other, even when running 32-bit applications; this requires specific support in virtualisation environments and assistance from the host operating system (at least, on 64-bit PC operating systems). OS/2 1.x relies on triple-faulting to leave protected mode, and OS/2 in general uses call gates for system calls, both of which require specific handling in the virtual machine manager and are much easier to implement with hardware assistance. (Other operating systems also use call gates, so that’s not an OS/2-specific requirement.)
All this is specifically a problem when virtualising on x86 without hardware virtualisation (AMD-V and VT-x). The goal when virtualising is to set the host environment up such that the guest can run directly on the CPU, with no translation, but with all privileged operations intercepted by the “hypervisor”. “Software virtualisation” on x86 is deficient for these purposes, and requires a lot of work in the hypervisor to make up for the architecture’s deficiencies. Hardware-assisted hypervisors can run OS/2 without difficulty, relying on the hardware to provide a complete protected-mode virtualisation; as can emulators (such as Bochs and QEMU in non-KVM mode), relying on their CPU emulation to simulate the protected-mode environment. Keith Adams’ and Ole Agesen’s 2006 paper, A Comparison of Software and Hardware Techniques for x86 Virtualization, explains the complexity of x86 virtualisation in detail.
answered Sep 6, 2021 at 13:22
-
3Dev: "So, which arcane features we'll use on our new SO?" Tech Lead: "Yes."– T. SarSep 6, 2021 at 16:04
-
28@T.Sar they weren’t arcane features when OS/2 was in development ;-). Sep 6, 2021 at 16:07
-
13In fact they were remarkable features! The protection system of Multics - more than the protection system of Multics - on a single chip microprocessor! (The GE645/H6180 systems did not have gates - call, task, or interrupt - and did all that through the segment hardware trapping on ring changes. I think gates were an improvement/optimization over that.)– davidbakSep 6, 2021 at 16:29
-
15@JörgWMittag this Super User Q&A gives some reasons why later OSs only used two rings. OS/2 was developed for 286s, where all four rings were equally useful (albeit still hard to use because of the limited number of segment descriptors); most later OSs were developed for 386s, where using more than rings 0 and 3 made less sense (if protection was provided by paging instead of segments). Sep 6, 2021 at 16:37
-
8That makes sense. It looks like it's a combination of the facts that the earlier OSs were either too simple to need it, or too portable to make use of it, and the later ones were designed for 386+ where it doesn't make sense. OS/2 seems to have been designed in a sweet spot where neither the 386 nor PowerPC existed, IBM didn't intend to port it, and they were also used to lots of hardware assistance through very powerful instruction sets from RS/6000 and their mainframes. Sep 6, 2021 at 16:44
LOADALLby any chance? (See LOADALL Strikes Again and How does the LOADALL instruction on the 80286 work?) That would have affected OS/2, at least 1.0 and 1.1, but not Windows 95 or NT. Do you have any references to Intel already having multi-CPU and multi-core in mind when they designed the 286?