21

Debuggers are carefully written programs that peek and poke other programs while they run. In retrocomputers, programs could use any part of the memory they could access.

So how did debuggers insert themselves into memory so they could be executed, without overwriting - or being overwritten by - the target program or its data?

  • 1
    This is a bit broad as it depends on the target. On a BBC they sat in sideways memory, on HPs there was a separate plug-in ROM. On others, e.g. Intel, they used separate hardware clipped onto the processor. – Chenmunka Jun 1 '16 at 12:15
  • 1
    What makes you think they did not insert themselves into memory? In many cases, they are already there, and you can't debug unless you have prepared something, where something could be for example inserting an exception handler. The OS may already do that. This question is not answerable without giving an example of a system that acts in the way you describe. – pipe Jun 1 '16 at 12:32
  • 1
    @pipe I know there were different ways of this being done. But I'm pretty sure it's answerable. – wizzwizz4 Jun 1 '16 at 12:36
  • except for hardware emulators they did fit in memory - but we're talking about earlier, much more primitive, less capable debuggers - not visual studio or eclipse. Here, for example, is the manual for the PDP-8's DDT debugger where you can see, on page 2-2, that it fit in ~2350 12-bit words (PDP-8 maximum memory: 32K words). But small and limited as it was we got a lot done with it! – davidbak Jun 1 '16 at 18:04
  • 1
    @RuiFRibeiro Tags that are synonyms are treated as the same. Although debug was there, the system will treat it as debugging when searching through stuff. The edit you made does help to keep things consistent though. – wizzwizz4 Mar 31 '17 at 15:42
28

Without hardware support, there is no way for a debugger to protect itself from the program being debugged. A debugger needs to protect its code and working state, as well as any hooks it's set up for its debugging (e.g. single-stepping interrupts); but in typical "retro" systems (8-bit machines), a debugger would just insert itself into memory and hope for the best.

Debuggers hosted on the same system as the code being debugged, and implemented entirely in software, would only be able to rely on the cooperation of the program being debugged and the person doing the debugging. Most debugging isn't adversarial: the code being debugged isn't actively taking measures against the debugger, so the debugger doesn't need to actively protect itself. (If a bug destroys the debugger as well as the program being debugged, well...) The debugger can offer configuration options to the end-user to facilitate things of course, but there will always be limitations that the end-user will have to live with (using a debugger leaves less RAM for the program being debugged, it might use some interrupts that the program would want for itself, etc.) — all told though the gain from having a hosted debugger makes it worthwhile.

Some early debuggers used extra hardware, without special CPU support, to reduce the level of compromise:

  • hosting the debugger in a cartridge means the code can't be touched
  • a cartridge can also provide extra RAM, which means the debugger's working state doesn't reduce the amount of memory available (at least, on systems with less RAM than their address space)

but, without a MMU, all this still happens in the same address space (even if banking is involved) so it's only reliable in non-adversarial debugging.

Adversarial debugging, or any form of complex debugging really (kernel debugging etc.) requires hardware support. This can take a number of forms:

  • a MMU to provide memory and I/O protection
  • CPU support for some level of virtualisation (e.g. V86-mode on the 386)
  • outside hardware, using a second computer for debugging — this was very common, in a variety of forms (a small debugging shim running on the host and communicating with the debugger via a serial port, the network etc.; ICEs clipped to the CPU, or replacing the CPU...)

Hosted debuggers, especially for adversarial debugging, only really took off with the arrival of MMUs and CPU-based protection; e.g. Turbo Debugger 386, SoftICE... Multi-host debugging is still used in many cases, e.g. embedded development, development on mobile phones (using emulators or real devices), kernel debugging...

There were some systems where the whole operating environment was constantly debuggable, e.g. Lisp Machines, but again that's non-adversarial debugging in most cases.

| improve this answer | |
  • 1
    +1 for "If a bug destroys the debugger as well as the program being debugged, well..." – Insane Jun 1 '16 at 19:18
  • +1 a very well thought answer; however I do beg to disagree and will write my own answer. – Rui F Ribeiro Mar 28 '17 at 19:15
19

A debugger that runs inside the debugged machine is a program, so it does need memory.

Sometimes the debugger is loaded as a ROM cartridge, usually with its own RAM, so it doesn't need to take any RAM from the running program. This is the case with, for example, the Action Replay modules for the Amiga.

Sometimes, it's a regular program that takes some RAM, but it can be reallocated at load time so if you know in advance that the program to examine won't use a certain block of RAM, you can load the debugger there. This is the case with, for example, the MONS 3 debugger for the ZX Spectrum

Sometimes, the debugger cannot find any "normal" RAM to execute from, so there are debuggers that load themselves into the screen memory, like this one:

Gray display with black garbage in the top third

Yes, that's a program. You can try to dump this screen into a ZX Spectrum emulator screen and do a PRINT USR 16384 to execute it :)

The only kind of debuggers that don't really need any memory at all from the debugged machine are the so called ICE (In Circuit Emulators), which either take the place of the main CPU, emulating its behaviour and reporting data to a external machine, or are logic analyzers that can decode bus cycles and present you a disassembly dump of real instructions actually executed in real time by the machine being debugged.

| improve this answer | |
7

I developed Logo for the Commodore 64, based on work we did at MIT for the Apple ][ and TI 99/4. Apple debugging was done with the ROM, via assembled-in breakpoints. For the C64, Andy Finkelstein at Commodore ordered me a 6510 CPU with an extra pin that signalled the I/D line, and a clamp-on connector that led to a Nicolet-Paratronics logic analyzer, connected to a PET with a disassembler in BASIC. I was having trouble with usage of page zero registers, both by the ROM and by Logo, which deferenced Nil (empy list [] in Logo), but on the 6510 locations 0 and 1 were the parallel port to peripherals such as the 1541 disk.

So, I could set a breakpoint when dinner code wrote or even read an address. What's even better is that I could set the breakpoint to start up to 255 bytes before the errant access occurred. It was the only time I could set a breakpoint in the past before an error occurred. And yes, it was more miraculous that the Lisp Machine debugger!

| improve this answer | |
3

On 8-bit machines any debugging support would just be a normal program in memory, and the application could accidentally overwrite the debugger. To single step through a program involved replacing some code in the application with a call or software interrupt that returned control to the debugger. It was all too easy to get a breakpoint in the wrong place and completely lose control of the debugger and the application.

Later on (in the early PC era), this principle was still the same. MS-DOS had a DEBUG application, and this would load in memory first, in turn loading the application on top of itself higher in memory. The 8088/8086 chips had no way to protect one segment of memory from another, so MS-DOS was not designed to manage memory to protect applications from each other. Consequently it was quite possible for the application under test to accidentally clobber the debugger and prevent it from working properly. Sometimes the machine just locked up and you just had to start back at the beginning.

| improve this answer | |
  • 1
    By the way, this was used as means for copy protection: the program under test could redirect vectors 01 and 03 to its own code, so the moment the TRACE bit was set by the debugger to allow stepping into the code, the redirected routine was called, trashing the debugger. I actually used that feature to implement a self-decoding routine: an encrypted program would set vector 01 to a routine that would generate a pseudo-random byte and would XOR it with a memory byte pointed by a previously setted register. At one point, I just toggle the TRACE bit to start the desencription process. – mcleod_ideafix Jun 4 '16 at 0:02
  • 2
    Actual what you describe is only one of at least two major approaches used in the 8-bit era. Many 80's era budget personal computers did rely on a a debugger loaded as an ordinary program into RAM, which would locate itself in a distinct area from the program being debugged. But a lot of earlier systems or those intended for embedded or system development instead had a ROM Monitor - effectively a very similar program, but ever present in EPROM rather than loaded into RAM as needed. – Chris Stratton Jun 6 '16 at 20:36
2

A debugger may be able to defend his integrity and memory space even in arcane architectures.

Granted that in the most common implementations, where you want to save speed and memory, and simplify the debug code, you are dependent on hardware protections provided by more advanced architectures.

The software can very well have anti-debug measures built-in for known debuggers or accidentally overwrite it in the course of loading or even in the normal operation of the program.

However, if you are willing to sacrifice memory and speed, it is rather easy to do "emulation" in the same CPU/interpreting critical commands and running others in a controlled area, at least more easy than implementing an emulation in a foreign architecture, as you are implementing actual arithmetic and bit operations using the native machine code instructions .

Taking the ZX Spectrum as an example, think it as an emulator for Z80 written in Z80 binary code just for the purpose of having an advanced debugger - think of it like pretending Z80 code is a foreign CPU, or as p-code and interpreting it (similar to how a BASIC program works).

To be fair, before I wrote my own Spectrum emulator for Windows, I was toying around for long about writing a native Z80 interpreter/emulator in the actual Z80 machine as a proof of concept, and even wrote the basics of this idea for debugging in a real ZX Spectrum 48K behaviour of undocumented Z80 opcodes for my emulation.

Even more interestingly yet, in machines like the TC/TS 2068 where you can control RAM paging from the 48K mode, you might get away with debugging full 48K programs if you get your paging code right.

Actually with such scenario you might even page in a screen with the debug interface and at the when running the commands write to the actual addresses/pages of the "real" screen.

Such a tool, besides the obvious debugging capabilities, would perhaps be more interesting as an educational tool if used contemporarily before emulation development systems in more advanced architectures became common place to hobbyists.

You might even extend such a tool with help of hardware, where you load a very protected piece of software (a turbo loader with the Alkatraz protection for instance), and later on take control of it with a NMI interrupt, passing control to such debugger in another page of RAM.

P.S. The comment of TS/TC 2068 might also apply to Spectrum +2 hardware, never had one so not sure.

The concept is also not new, and it was implemented in the past, albeit in more advanced systems. One of the oldest (from 1994) and better example I know of, is Bochs

Bochs (pronounced "box") is a portable IA-32 and x86-64 IBM PC compatible emulator and debugger mostly written in C++ (...) It supports emulation of the processor(s) (including protected mode), memory, disks, display, Ethernet, BIOS and common hardware peripherals of PCs.

Many guest operating systems can be run using the emulator including DOS, several versions of Microsoft Windows, BSDs, Linux, Xenix and Rhapsody (precursor of Mac OS X). Bochs runs on many host operating systems, including Android, iOS, Linux, Mac OS X, PlayStation 2, Windows, Windows Mobile.

Bochs is mostly used for operating system development (when an emulated operating system crashes, it does not crash the host operating system, so the emulated OS can be debugged) and to run other guest operating systems inside already running host operating systems. It can also be used to run older software – such as PC games – which will not run on non-compatible, or too fast computers.

| improve this answer | |
  • That's an interesting idea, but wouldn't it be painfully slow on the computers in question? Did any debuggers work like this when those systems were still current? – Stephen Kitt Mar 28 '17 at 21:13
  • Probably painfully slow...you had a couple of contemporary utils that produced p-code, namely a util called *Blaster*(?), and whilst it was not machine code, it was acceptable. Forget about it for real time operations like loading or saving from tape. Even now it would be an interesting exercise to write such a thing. Even in the folklore, you can read about Bill Gates writing a Z80 simulator to write the original BASIC in very primitive systems (compared to today). However, let´s put in perspective such a contraption would be only used for debugging code... – Rui F Ribeiro Mar 28 '17 at 21:52
  • ...in very specific situations or for teaching purposes.As I said, I used something that could be the basis for such thing to debug undocumented opcodes in a time when they were not so well documented. Without exaggeration, whilst I duplicated worked of previous people in a non-Internet world, I also helped to document a couple of things in the Speccy emulation scene that nowadays people take from granted.Old comp.sys.sinclair FAQs used to have the names of contributors of several tidbits. Granted that blaster helped itself of ROM routines, but people writing early emulators is a best example. – Rui F Ribeiro Mar 28 '17 at 21:52
  • Alas, if memory does not fail me, the C and Pascal compiles from Hisoft, were not compilers per se, but translated the Pascal and C code to p-code and then interpreted it. – Rui F Ribeiro Mar 28 '17 at 21:55
  • 1
    I'm familiar with the concept, I've used Bochs quite a bit (and some older proprietary same-CPU simulators too). I was just curious about the feasibility on 8-bit CPUs. Your use of a simulator to debug undocumented Z80 opcodes sounds very interesting indeed... – Stephen Kitt Mar 29 '17 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.