The Intel 80386 CPU didn't have an on-board x87 FPU (maybe with the exception of some non-Intel clones). It was, however, able to use either a 80287 or 80387 as an external FPU. When the x87 FPU accesses memory, the CPU makes the necessary privilege checks and generates an exception if the access is illegal (not enough privilege, page/segment not present etc.).
On CPUs with an on-board FPU, the exception generated is the General Protection Fault (interrupt 13), just like all other non-FPU illegal accesses.
On CPUs without an on-board FPU (like the 80386), the exception generated is the Coprocessor Segment Overrun Exception (interrupt 9), unlike all other memory accesses which use the GPF.
The Intel i386 manual states the following about how INT9 should be handled (emphasis is mine):
The addresses of the failed numeric instruction and its operand may be lost; a
FSTENVinstruction does not return reliable numeric coprocessor state information. The coprocessor-segment-overrun exception should be handled by executing a
FNINITinstruction (i.e., a
FINITinstruction without a preceding
WAITinstruction). The return address on the stack might not point to either the failed numeric instruction or the instruction following the failed numeric instruction. The failed numeric instruction is not restartable; however, the interrupted task may be restartable if it did not contain the failed numeric instruction.
What I understand from this is that if the FPU makes an illegal memory access then its entire state cannot be recovered. Am I actually getting this right?
What happens if the illegal access happened, for example, because the memory was swapped to disk? Normally the OS would load the missing page to memory, put it back in the virtual address space and continue execution. However, with this behavior, it is impossible to do that because ‘The failed numeric instruction is not restartable’. You cannot get the FPU state either, all that can be done is to
FNINIT and kill the innocent application.
I also don't understand this statement very well: ‘The return address on the stack might not point to either the failed numeric instruction or the instruction following the failed numeric instruction’. Does this mean that the return address is undefined and all the OS can do is kill the application and return to some other task by crafting a return address on the stack?