16

It's pretty well documented how to get the 286 from real mode to its 16-bit protected mode.

It's also widely stated that the 286 had no officially supported way of going back from protected mode into real mode without losing its current state. The "obvious" way to do it was pretty much a no-op.

However, developers still managed to coax the 286 into switching back to real mode from protected mode to access real-mode-only services while protected-mode software was and remained running.

What techniques were used to make the switch back to real mode?

Note: I am specifically asking about the 286 here; I am not interested in answers pertaining to the 386 or newer CPUs.

  • 4
    There was always an official and documented way of going back to real mode while in protected mode, resetting the processor. The trick was doing this in a way that would be useful for operating systems (and the BIOS services that allowed accessing extended memory from real mode.) – Ross Ridge Aug 6 at 16:47
  • Intel designed the 80286 to boot in real mode, use the basic hardware to things up, and then go into protected mode AND STAY THERE, NEVER TO COME OUT, unless power was cycled. – John R. Strohm Aug 12 at 18:37
31

I’m aware of three main techniques used to exit protected mode on 286s; they all involve resetting the CPU, they vary on how that is done:

  1. using the keyboard controller (which IBM endowed with the ability to reset the CPU);
  2. using a chipset-specific “fast reset”;
  3. using a triple fault.

There’s a detailed discussion of the first and last approaches on Robert Collins’ site. To reset using the keyboard controller, you write 0xFE (the code for the shutdown command) to the 8042 status port (0x64). To reset using a triple fault, you set the limit of the IDTR to 0, and generate an interrupt; since the segment is now invalid, the CPU faults, but that also involves an invalid segment, so the CPU faults again twice before resetting.

When the CPU resets, it starts running from near the top of memory, in the BIOS; the BIOS checks a code in the CMOS, or a value in RAM, to determine whether it’s running a full boot-up or it’s recovering from a protected mode exit. To set this up, store the restart address (the real mode address your program will resume from) in the DWORD at 0x0040:0x0067, and set the CMOS shutdown type to 5 (see SETPM_RET_ADDR and SET_SHUTDOWN_TYPE for details; these run in real mode before the switch to protected mode, but the setup can also be done in protected mode). After resetting, the program needs to restore the CPU state properly (in particular, it needs to reload all the segment registers and the stack pointer); see Robert Collins’ documentation for details.

  • 2
    Why does setting the shutdown type needs to happen before entering protected mode? In the code you link to, it seems to be a simple matter of a few OUT instructions; wouldn't it work just as well to do them from protected mode? – Henning Makholm Aug 6 at 22:22
  • @HenningMakholm Can you actually set the return address for the reset if you're already in protected mode? That might be the real problem, rather than setting the shutdown type itself. There are different ways of system shutdown that are controlled by the shutdown type, so it'd be a bit weird if you had to switch to real mode before doing a soft reset, for example. – Luaan Aug 7 at 8:34
  • 1
    @HenningMakholm that’s a good question, and I didn’t think about it when writing the answer! It’s certainly easier to set up the exit from PM while still in RM, but it appears not to be strictly necessary. Setting it up before entering PM does help with handling any involuntary triple faults though ;-). – Stephen Kitt Aug 7 at 8:53
  • @Luaan: The particular code for setpm_ret_addr in Stephen's link will only work in real mode, but nothing would prevent sufficiently privileged code in protected mode from setting up a segment descriptor that will let allow it to write that part of physical memory. – Henning Makholm Aug 7 at 10:39
  • 1
    Larry Osterman (engineer at Microsoft) describes on his blog the triple fault command they discovered for the 286. – Peter Cooper Jr. Aug 7 at 18:44
9

The trick is that resetting the CPU leaves the RAM intact (unless the system designers have done something that will clear some or all RAM on a reset). So if you leave appropriate information in the RAM to be read by the (real-mode) startup code, it can detect that before the reset a "message" was left to it to do whatever needs to be done in real mode before going back to protected mode. (Re-entering protected mode would also generally use the protected mode descriptor tables and other information also left intact in RAM, though of course any setup for other devices (such as interrupt controllers) that were also reset also needs to be redone.)

This is documented in a bit more detail on Wikipedia.

0

Fastest way was not to go to protected mode in the first place. Using the undocumented LOADALL opcode the CPU state can be freely set, except it can't be used to return from protected mode to real mode. But actually most compatible way was to cause a triple fault when return from protected mode was needed.

  • 5
    How does this answer the question? – edc65 Aug 7 at 9:41
  • How did you cause a triple fault? – Chenmunka Aug 8 at 9:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.