20

It's pretty well documented how to get the 286 from real mode to its 16-bit protected mode.

It's also widely stated that the 286 had no officially supported way of going back from protected mode into real mode without losing its current state. The "obvious" way to do it was pretty much a no-op.

However, developers still managed to coax the 286 into switching back to real mode from protected mode to access real-mode-only services while protected-mode software was and remained running.

What techniques were used to make the switch back to real mode?

Note: I am specifically asking about the 286 here; I am not interested in answers pertaining to the 386 or newer CPUs.

Improve this question
2
  • 7
    There was always an official and documented way of going back to real mode while in protected mode, resetting the processor. The trick was doing this in a way that would be useful for operating systems (and the BIOS services that allowed accessing extended memory from real mode.)
    – user722
    Commented Aug 6, 2019 at 16:47
  • 3
    Intel designed the 80286 to boot in real mode, use the basic hardware to things up, and then go into protected mode AND STAY THERE, NEVER TO COME OUT, unless power was cycled.
    – John R. Strohm
    Commented Aug 12, 2019 at 18:37

3 Answers 3

Reset to default
39

I’m aware of three main techniques used to exit protected mode on 286s; they all involve resetting the CPU, they vary on how that is done:

  1. using the keyboard controller (which IBM endowed with the ability to reset the CPU);
  2. using a chipset-specific “fast reset”;
  3. using a triple fault.

There’s a detailed discussion of the first and last approaches on Robert Collins’ site. To reset using the keyboard controller, you write 0xFE (the code for the shutdown command) to the 8042 status port (0x64). To reset using a triple fault, you set the limit of the IDTR to 0, and generate an interrupt; since the segment is now invalid, the CPU faults, but that also involves an invalid segment, so the CPU faults again twice before resetting.

When the CPU resets, it starts running from near the top of memory, in the BIOS; the BIOS checks a code in the CMOS, or a value in RAM, to determine whether it’s running a full boot-up or it’s recovering from a protected mode exit. To set this up, store the restart address (the real mode address your program will resume from) in the DWORD at 0x0040:0x0067, and set the CMOS shutdown type to 5 (see SETPM_RET_ADDR and SET_SHUTDOWN_TYPE for details; these run in real mode before the switch to protected mode, but the setup can also be done in protected mode). After resetting, the program needs to restore the CPU state properly (in particular, it needs to reload all the segment registers and the stack pointer); see Robert Collins’ documentation for details.

Improve this answer
14
  • 3
    Why does setting the shutdown type needs to happen before entering protected mode? In the code you link to, it seems to be a simple matter of a few OUT instructions; wouldn't it work just as well to do them from protected mode?
    – hmakholm left over Monica
    Commented Aug 6, 2019 at 22:22
  • 1
    @HenningMakholm that’s a good question, and I didn’t think about it when writing the answer! It’s certainly easier to set up the exit from PM while still in RM, but it appears not to be strictly necessary. Setting it up before entering PM does help with handling any involuntary triple faults though ;-).
    – Stephen Kitt
    Commented Aug 7, 2019 at 8:53
  • 2
    Larry Osterman (engineer at Microsoft) describes on his blog the triple fault command they discovered for the 286.
    – user616
    Commented Aug 7, 2019 at 18:44
  • 1
    @Alexander in this setup the memory contents are preserved, only the registers need to be restored; the CPU reverts to its startup state on reset, but nothing else changes.
    – Stephen Kitt
    Commented Aug 7, 2019 at 21:20
  • 2
    @Alexander RAM would generally be overwritten as a side effect, not as a deliberate choice. If you're curious, do consider posting a separate question so the issue can be answered in more depth.
    – user
    Commented Aug 8, 2019 at 6:41
12

The trick is that resetting the CPU leaves the RAM intact (unless the system designers have done something that will clear some or all RAM on a reset). So if you leave appropriate information in the RAM to be read by the (real-mode) startup code, it can detect that before the reset a "message" was left to it to do whatever needs to be done in real mode before going back to protected mode. (Re-entering protected mode would also generally use the protected mode descriptor tables and other information also left intact in RAM, though of course any setup for other devices (such as interrupt controllers) that were also reset also needs to be redone.)

This is documented in a bit more detail on Wikipedia.

Improve this answer
1

Fastest way was not to go to protected mode in the first place. Using the undocumented LOADALL opcode the CPU state can be freely set, except it can't be used to return from protected mode to real mode. But actually most compatible way was to cause a triple fault when return from protected mode was needed.

Improve this answer
3
  • 5
    How does this answer the question?
    – edc65
    Commented Aug 7, 2019 at 9:41
  • How did you cause a triple fault?
    – Chenmunka ♦
    Commented Aug 8, 2019 at 9:30
  • @Chenmunka: Triple fault is what happens when you deny access to the interrupt vector table and raise an interrupt.
    – Joshua
    Commented Feb 11 at 4:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .