It's been a while, but I've read in a system programming book that you could switch your Intel 80286 CPU from the normal real mode to a more powerful protected mode. I clearly remember that they said it was impossible to switch back though, unless you fully restarted the computer.

Why was this (made?) impossible? Was it a bug, intention by the DOS developers or a hardware restriction of the CPU?

  • Exactly what happens when CPUs are created by programmers (you remember, 8086 was architected by a pure programmer. Probably 80286 still was done the same). :) – lvd Apr 8 '18 at 15:56

This was intentional so that the CPU would support secure operating systems. In a secure operating system with rigorous memory access protections you could not allow any software - user or kernel extension or driver - to switch back to the full freedom of real mode.

They had a lot of interesting memory management hardware on the '286: rings and call gates - swiped nearly directly from the most secure hardware/software platform of the time: Multics. And they had new extensions too: task segments and task gates for multiprocessing.

These features never got used as they wanted. One major reason was that due to compatibility with all the peripherals - including disk controllers and graphics hardware that in those days were nowhere near sufficiently standardized - OS and other software that needed computer model/manufacturer independence needed to use the BIOS to access peripherals - and the BIOS was real mode - and the hacked-up switch to real mode (as discussed in other answers) was slow. Another major reason was that calling through call gates, and using hardware tasks, was much much slower than just implementing normal procedure calls and concurrent threads. Hardware task switching in particular was several times slower than just saving/restoring register context with normal instructions. So no OS software got written for these special modes.

These capabilities were also available when 32-bit processors were made - starting with the '386 (and they're still there in the x86 architecture to this day) - but with 32-bit addressing it turns out it is much easier to use paging hardware (address translation tables and so on) to get OS security - also much higher performance. No 32-bit OS used the stuff: Windows, OS/2, and all the Unixes used paging for process isolation and security.

I was actually saddened by this back in the day: I loved Multics, and the Multics architecture. The Intel designer's hearts were in the right place, and it was brilliant work sticking all that stuff in a commodity microprocessor, but it turned out to be the wrong solution for the problem. Operating system engineering had progressed past the Multics days and all that special hardware just did not lead to an economic solution.

  • 1
    Unfortunately, I think the designers of the 80286 and 80386 failed to recognize what was good about the 8086 segmented architecture: it allowed a 16-bit CPU to access objects up to 65536 bytes which were located on 16-byte boundaries, without having to add per-object overhead. Use of the 8086 architecture was a bit clunky because it had one too few general-purpose segment registers and couldn't do arithmetic on segment registers or even use load-immediate with them [were it not for that, exe files could easily use DS for "whatever" and cheaply reload with a program's main segment], but... – supercat Apr 9 '18 at 0:45
  • 1
    The ring system IS used by modern OSes... not all of them though. – rackandboneman Apr 12 '18 at 22:34
  • 3
    Ring 0 and 3, sure, but calling that "using the ring system" is a stretch. The intention was for the OS to be multi-layered. As used, the rings are just an implementation detail of how OSs set up a wall between user and privileged modes in x86/x64. – Euro Micelli Apr 13 '18 at 4:28
  • 4
    OS/2 is notorious for causing virtualization software developers headaches due to its additional use of ring 2, not just 0 and 3. – rackandboneman Apr 13 '18 at 7:49
  • 2
    @rackandboneman - I did not know that but Google tells me they put device drivers in ring 2. Good for them! Sorry it didn't work out ... – davidbak Apr 13 '18 at 14:56

My guess is that it was merely a design decision based upon the assumption that once a protected mode OS is started, there is no need to go back. Most microprocessors at that time already booted in its most privileged mode and had at least two levels of protection. The 80286 had to boot in real mode to keep compatibility with DOS and I think they thought DOS would reduce itself to just a minimal procedure to boot the main OS.

It seems that Intel engineers didn't realize that DOS was going to live for about ten years after launching the 286, and software engineers along with motherboard manufacturers would figure out a way to switch the CPU back to real mode in order to call DOS services from a protected mode program (who decided that the keyboard controller is a good place to put a register with a bit to reset the CPU and another one to enable CPU addresses beyond 1MB?)

By the time the 80386 was produced, they added the feature to switch the CPU back to real mode, and the dirty trick used with the 80286 -to reset it with a magic number at a certain memory location so the BIOS could read it and jump to some predefined code to resume operation- was not needed any more.

Then, the undocumented LOADALL instruction was spoiled on the USNet, which could allow not only switching from and to real mode, but allow non standard CPU states, such as the so called "unreal" mode, which allowed a real mode DOS program to access memory beyond the 1MB barrier.

This article from the OS/2 museum discuss the use of LOADALL to switch back from protected mode, and how Microsoft used it in HIMEM.SYS to allow fast above-1MB-memory copying without having to leave real mode.

  • 3
    See: en.wikipedia.org/wiki/Real_mode#Switching_to_real_mode One of the main drivers for these hacks was getting access to in13 BIOS calls, which were real mode only. – user12 Apr 22 '16 at 14:53
  • 1
    "Should", but often didn't. – user12 Apr 22 '16 at 15:03
  • 2
    IBM's PS/2 systems had an ABIOS, which was a protected-mode BIOS designed to be used with OS/2, and allowed the system to stay in protected mode (except for the DOS "penalty box"). – Stephen Kitt Apr 27 '16 at 19:16
  • 2
    @mcleod_ideafix It's a nice theory, but given the fact that hardware manufacturers at the time didn't really see much point in supporting any operating system other than DOS and that the interfaces to their hardware were often entirely undocumented, the fact that cards often had BIOS extension ROMs onboard to make them work with DOS was about as good as you were going to get, so if you wanted to make another OS work with them, you needed that OS to be able to interface with the BIOS. – Jules Jun 2 '16 at 19:32
  • 2
    Someone had to start departing MS DOS, and Intel tried to support other OSes. In fact, the very datasheet of the 80286 starts with "The 80286 is an advanced, high-performance microprocessor with specially optimized capabilities for multiple users and multi-tasking systems...". Intel assured that the 80286 was compatible with existing legacy real mode programs, but looked towards MS DOS and saw XENIX, UNIX, etc, and realized that some sort of hardware assisted protection was needed to support them. – mcleod_ideafix Jun 2 '16 at 22:46

Just speculating here, but it might have been a product decision to encourage writing code for protected mode. It's also possible it was a combination of technical difficulties and product priorities.

The CPU is put into protected mode by setting the PE bit in MSW using the LMSW or LOADALL instructions. Clearing the PE bit has no effect using either of those instructions, thus it is not possible to switch back to real mode. The fact that LOADALL allows undocumented behaviours like "unreal mode" (by loading values into descriptor caches to access memory outside of 1MB) and even allows you to put the CPU into an unusable state by loading nonsensical values, but disallows clearing the PE bit makes it likely it was a deliberate product decision. OTOH, LOADALL is undocumented, so maybe there really was a technical problem.

Just like switching from real to protected mode, going back to real mode is a carefully choreographed dance that needs to make sure all memory accesses go to defined memory locations (especially instruction fetches). The documentation has a strange passage that says "After executing LMSW instruction to set PE, the 80286 must immediately execute an intra-segment JMP instruction to clear the instruction queue of instructions decoded in real address mode" hinting that there is more to it than just setting or clearing a bit somewhere. It probably wasn't completely trivial to implement (not very hard either, but it definitely would have had a cost if you're counting transistors or microcode ops. The 80286 had ~134,000 transistors, and using a couple 100 transistors to implement an unnecessary feature would probably have meant dropping other, deemed more important features). Since they didn't anticipate a need for it, maybe they weren't willing to pay the "silicon tax".

There are still ways to get back to real mode by resetting the CPU without restarting the computer. On the PC, this could be done by putting a magical value into a special memory location to prevent the BIOS from reinitializing the computer after a reset, and then causing a reset through the i8042 keyboard controller which would externally reset the CPU, or by generating a triple fault (faulting in a fault handler, usually by invalidating the IDTR and causing an interrupt).

Using the triple fault is usually much faster. Going through the keyboard controller can take almost a millisecond, the triple fault only a couple hundred microseconds. Even with the triple fault, the reset circuitry is still external, but it doesn't have to be processed by the very slow i8042 (some PCs of the time "short-circuited" the reset bit in the 8042 port to directly reset the CPU without going through the i8042, in that case, there's no noticeable difference).

The reasons to switch back to real mode are all due to technical debt, you wouldn't need it in a clean system that's designed for protected mode from the ground up. In reality, people kept writing software for real mode and were unwilling (or unable) to quickly port stuff like device drivers to protected mode, and the 2 don't mix. Maybe Intel was trying to "encourage" people to port their drivers to protected mode by making it necessary, but as we know, this didn't work out.

Intel didn't make that mistake again, and the 80386 fixed the glitch both by allowing mode switches in all directions, and by adding a virtual 8086 mode that was essentially a protected mode that looked like real mode to the application.

  • 1
    It would had a cost.... or not. The 80286 is a microprogrammed arquitecture, and there was already a means of altering the MSW using SMSW. Only that it didn't allow to reset the PE bit. In addition, they implemented LOADALL, which is way much sophisticated than SMSW (and I believe it could put the micro back into real mode) – mcleod_ideafix Jun 24 '16 at 13:57
  • I'm not sure there was a technical reason for not allowing the switch back to real mode, it might really have been a product strategy thing. Editing the answer to mention the microcode, MSW and LOADALL – Rico Pajarola Jun 25 '16 at 14:34
  • @RicoPajarola: Wouldn't the fact that real mode can switch segments faster than protected mode be an argument in favor of using the former except in places where code needed the features of the latter? – supercat Apr 10 '18 at 18:58
  • 2
    Don't know where else to stick this story so I'll stick it here: In those days one use of RAM "above 640K" was for a RAM disk (RAM cache). E.g., IBM VDISK.SYS, though several manufacturers shipping DOS had variants. HP had one. They'd frequently use protected mode to access that memory - switch DOS to protected mode, do the ramdisk stuff, then use the 8042 hack to switch back. Alsys' Ada compiler for the x86 - which I worked on at Alsys - was a DOS compiler which ran in protected mode (and shipped with a 4Mb card!) - also it also used it the 8042 hack. But, in a horrible brain fart ... – davidbak Apr 13 '18 at 3:46
  • 1
    ... the HP engineers, on their version of the IBM PC AT, made the BIOS buzz the PC speaker each and every time the switch back to real mode was done! Arrggghhhh! We had to clip the wires to the PC speaker on our test lab HP PC and had to hope none of our customers were buying HP! (The BIOS, of course, was in ROM.) (Just one minor - but annoying! - example of the incompatibilities between various PC BIOSs at the time.) – davidbak Apr 13 '18 at 3:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.