33

Back in the Windows 95/98 days, the "illegal operation" dialog was (from here):

A Windows 9x application crash dialog box window with the title ‘Setup’. The first paragraph of the text inside reads ‘This program has performed an illegal operation and will be shut down.’, and the second reads ‘If the problem persists, contact the program vendor’. There are two buttons to the right, reading ‘Close’ and ‘Details>>’.

but sometimes, in that dialog, it would say "Quit all programs, and then restart your computer" (from here):

A similar dialog box as before, but now the title is ‘<unknown>’, and the second paragraph reads ‘Quit all programs, and then restart your computer. If the problem persists, contact the program vendor.’. The ‘Details>>’ button on the right has been clicked to reveal an execution state dump below, where the first line reads ‘<UNKNOWN> caused an exception c0000006H in module <unknown> at 0000:bff86eac’

What would cause the "Quit all programs..." to appear in the "illegal operation" dialog? (that is, which kind of illegal operation? I assume it managed to corrupt something deep within Windows).

I looked at an old copy of MSDN and couldn't find much, maybe I was using the wrong keywords.

Improve this question
3
  • 10
    I imagine it depends on whether the fault occurs in application memory or 'windows' memory. Note that the EIP in the fatal-to-system example is up there in high address space, where the Windows DLLs are mapped. (That's pretty much normal for fault handling - the recovery action depends on whether it's localizable to a process).
    – another-dave
    Aug 10 at 1:27
  • 7
    For what it's worth, c0000006H is STATUS_IN_PAGE_ERROR. See this 2008 blog post by Raymond Chen for more. It's on the more serious side as exception codes go, but I don't know whether that affects the message that's generated.
    – benrg
    Aug 10 at 1:33
  • 7
    From the title line of teh requester, you can se that the first one points at teh program "Setup", the second one points at "<unknown>". That might also have been the reason for the second requester. If you don't know what program it is that gives the problem, you better turn everything off and reboot.
    – UncleBod
    Aug 10 at 6:24

1 Answer 1

Reset to default
37

In case of faults like "illegal instruction" or "privilege violation", the trap handlers in Windows examine the return address on the stack to try and locate the offending instruction. If that address points into the address space of a program, you'll see the first dialog that simply asks you to close the offending program, assuming nothing else has been compromised in Windows (this assumption is not always true, as you might have found. Actually, Windows can't really 100% know whether the offending program has not destroyed any important system data structures when it went bust).

The second dialog is shown when the return address cannot be associated with a specific program in menory, but rather points into a Windows DLL or other OS routines or, as in your specific example, points into something that was about to be paged in but can't. Here, Windows cannot easily find out what program has called this DLL or initiated the system call, so cannot point to a specific application and, more importantly, cannot assess the damage that might have been done. So it rather recommends to restart the system.

Improve this answer
4
  • 3
    How does it find the return address? Does it assume it's at [EBP+4], which would be the case in a function that used EBP as a traditional frame pointer? I guess back in 1995, compilers probably defaulted to that, if gcc -fomit-frame-pointer even existed yet.
    – Peter Cordes
    Aug 11 at 1:02
  • 1
    @PeterCordes EPB (or any compiler settings) are not involved when an x86 trap is called and the stack frame format is the standard x86 interupt stack frame: SP and IP of the offending instruction can be found there and, depending on the CPU mode, mapped to either a system or user process.
    – tofro
    Aug 11 at 6:02
  • 1
    Oh, you mean the exception return address. That makes a lot more sense, checking the location of the instruction that cause the #UD illegal-instruction fault, instead of the parent of the function containing it! I thought that sounded weird, but hey, it was Windows so I didn't think too hard about it. xD
    – Peter Cordes
    Aug 11 at 6:09
  • 2
    Programming mistakes (or malices) that damaged OS owned in memory structures were trivial to write in Windows 95/98 - usually, passing a pointer to something else than what it was supposed to point to to API functions that wrote through the pointer did the trick. Eg, mixing up GDI handles and window handles ... kablam!
    – rackandboneman
    Aug 15 at 20:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .