13

Lately I've been interested in how old machines work, in particular an NES. While there are quite a few resources on the basic operations and even some games that have been totally broken down byte by byte, there isn't much information on how this is done.

So far, I've found discussions on two different methods to accomplish this, but neither seems very efficient, so I'm curious to see if there is a better option. Firstly there's converting the hex ROM file into assembly code, but this is difficult to follow and deals with RAM operations, making it difficult to precisely where in the ROM the information is being drawn from. Secondly, you can go in and systematically delete or corrupt parts of the ROM file just to see the effects. I've personally had more success with this, but it takes forever, and is entirely guesswork and doesn't lead to a broader understanding of how the program actually operates.

Are these the only two viable options to doing this? Are there any patterns or behaviors I should be aware of? Thanks.

  • Welcome to Retrocomputing Stack Exchange. This is a really interesting question; I've tried to do this before but haven't got far. Are you looking for all data or just certain bits of data? – wizzwizz4 Apr 26 '17 at 6:01
  • 6
    That's a very broad question, and it basically amounts to "how is reverse engineering done". I guess everyone has a preferred way; I usually start out with an assembly listing, look at data, split it into routines, try to guess what some routines do, and then work my way up. With emulators, you have a lot more options, both for initial guesses, and finding out details. And yes, it takes forever. – dirkt Apr 26 '17 at 6:02
  • 2
    I agree with @dirkt. Another interesting option with games is to look for sprite data and other graphical data; they can be spotted quite easily in ROM dumps. – Stephen Kitt Apr 26 '17 at 8:18
  • @wizzwizz4 I've been searching for enemy behavior data, I've been looking at Kirby for NES and trying to find how enemies transfer their characteristics to Kirby. I've been able to find text and tile data easily enough, but determining subroutines is much trickier. – Aquova Apr 26 '17 at 13:21
  • Emulators enable powerful analysis, though not many do as much as they could. Here's a video of a feature I was playing with a few years ago in an Apple II emulator. A few other emulators have similar features, but I'm not aware of any NES ones. – Nick Westgate May 1 '17 at 23:54
16

I've done this partially with Commodore PET Space Invaders which I used as a test program to debug my Commodore PET emulator.

I used a disassembler to convert the program to assembly language and I then went through the code annotating it as I found out what it does. When I understood what a bit of code did, I would look for its entry point (my disassembler created labels for the target of every JMP and JSR) and rename the label to something useful. That would help me with other bits of code that called the piece I had just reverse engineered.

It helped that, having written an emulator, I was quite familiar with the memory mappings of the IO chips and certain important bit patterns. So, for instance, if I saw a bit of code writing a certain bit pattern to a certain address that I knew was mapped to one of the PIA ports, I knew it would be scanning the keyboard.

It also helped that I could run my emulator in trace mode (albeit very slowly) so every instruction is printed out as it is executed. So, for instance, when a space invader started appearing on the screen, I could look at the trace and see which bits of code were being executed.

It took some time and I basically suspended work on it when I found a bug that was caused by my emulator being too fast. I haven't figured out a way to make it run accurately at the correct speed yet.


To expand a bit, I used the disassembler (da65) from the cc65 tool chain. It has the handy ability to accept an "info file" so you don't need to annotate the generated source from the disassembler. So, for example, the first two bytes of a PET program file are actually the load address. I wrote the following annotation in the info file

RANGE { 
    NAME "loadAddress";
    START $03FF;    
    END   $0400;
    TYPE ByteTable;      
};

which make the assembler treat the first two bytes as data and not garbage assembler instructions.

The next bit was a one line BASIC program that calls the machine code routine that is the start of the program proper. Again, I forced the disassember to treat it as data

RANGE { 
    NAME "basicBootstrap";
    COMMENT "SYS(1039)";
    START $0401;    
    END   $040E;
    TYPE ByteTable;      
};

This time it also causes the disassembler to emit a comment that tells you what it is.

And then it's just a case of laboriously going through the code identifying the function of various bits. For example, I found a short loop that tested a certain bit on one of the PIA registers which I know is the vertical retrace, so it is clearly syncing with the monitor. I add this annotation

LABEL { 
    NAME "waitForRetraceLow";     
    ADDR  $09B0; 
    size $8;
    COMMENT "Wait for the screen to lower its retrace signal";  
};

and the disassembler creates a label for the routine as well as using the label in JSR and JMP from elsewhere.

2

I just answered a similar question of yours on stackoverflow.com which may help: https://stackoverflow.com/questions/43573269/how-does-a-6502-processor-transfer-data-between-rom-and-ram

I am not quite clear what you mean by "deals with RAM operations". Perhaps you could clarify?

Yes, reverse engineering is a long process. I am working on deciphering a games cartridge I wrote 35 years ago and am finding it difficult. Understanding what the code does is one thing but understanding why it does it is far more complicated and requires a good understanding of the console's hardware and the logical operation of the game.

Getting back to your real question though, somewhere in the code it must be accessing the ROM (and again I refer you to my response on stackoverflow.com). What I would do would be to search the disassembled code for any reference to the ROM area. So, for example if your ROM starts at $C000, run Find commands from your text editor for $C, $D, $E, $F. Hopefully that will locate something that might get you started.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.