Did any assemblers work like this?

Question

Once you get past the stage of manually entering numeric machine code, the most primitive programming tool is a machine code monitor. Intended primarily as a debugging tool, this works with a target program in memory as machine code, and can disassemble small chunks of the code in memory, and typically provides a simple assembler that basically does one instruction at a time, with little or no symbolic capability e.g. no branch target resolution. Perhaps the best-known example is the MS-DOS tool DEBUG.COM.

Past that stage, the next step up is a full-blown symbolic assembler. On 8-bit computers, these were the most commonly used commercial programming tools; I wrote one myself on the Commodore 64, back in the day. They worked the way modern assemblers do: store the source code in a text file, make two passes, or one pass with back patching.

An important limitation of these assemblers was memory. Source code as ASCII text is a rather bulky format; a large program would not fit in memory in this format alongside the assembler and the generated machine code, which meant (unless you could afford to cross develop from a 16-bit machine) waiting a long time for a build to run on floppy disk.

Looking back on that, if I were writing an 8-bit native assembler now, I think I would try a different approach, in between machine code monitor and full classical assembler: keep the program under development, in memory always in binary form (more compact than source text, much more compact than both source and binary), but also keep a comprehensive symbol table so that the IDE could disassemble on the fly into something that would give the illusion that you have source code in front of you. There would be some limits, e.g. each address could be given no more than one name, but it seems to me the memory saving would be well worth it.

So my question is: did any assemblers with the above design, ever exist?

Answer 1

...in between machine code monitor and full classical assembler: keep the program under development, in memory always in binary form (more compact than source text, much more compact than both source and binary), but also keep a comprehensive symbol table...

did any assemblers with the above design, ever exist?

Yes. In 1984 The Home Computer Advanced Course came with an assembler-debugger package on tape called CHAMP, with versions for the ZX spectrum, Commodore 64 and BBC Micro. Soon afterwards I 'unoffically' ported the Spectrum version to the Amstrad CPC. 26 years later I produced an updated version with more features.

When you type a line of source code into CHAMP's assembler it immediately converts it to machine code, with pointers and tokens to handle symbols and math operations. This format takes up about 1/3rd of the space of plain text, and 'assembles' much faster because the text has already been parsed. The source text is displayed by disassembling each opcode 'on the fly', but is still fast enough to get a reasonable scroll rate.

A lot of code is shared between the assembler and debugger, so the program itself is fairly compact for an 'IDE'. I put it in a ROM to use up less RAM and make it more crash-proof, and used the CPC's bank switching to keep the source code hidden when the executable is running. This enabled writing, assembling, and debugging large programs entirely in memory.

I also created a 'reassembler' which used CHAMP to disassemble itself into CHAMP source code. This is how I got past the patching stage when developing it on the Amstrad. To transfer the original program from the Spectrum to the Amstrad I built a custom parallel interface for both computers. Then I wrote a disassembler in BASIC and printed the result out on fan-fold paper, figured out what to patch to get the debugger working, then finally was able to use CHAMP itself to do the rest!

Answer 2

The LISA assembler for the Apple II kept the source file in a partially-processed form. LISA v2.x mostly just tokenized the opcodes, but v3+ kept symbols in a symbol table, tokenized arithmetic operations and operands, and did other funky things. Comments and lines that failed to parse were stored in their "raw" form. I think macros were as well.

The source code was kept this way on disk and in memory. This allowed it to use less RAM and assemble more quickly.

The only documentation I've been able to find on the format was the 6502/65816 source code itself. C++ code to decode the various versions can be found in CiderPress (search for "LISA Assembler" in the sources).

Answer 3

So my question is: did any assemblers with the above design, ever exist?

It may have done but I have never come across such an assembler. Two important features of good assemblers could not have been supported:

• comments
• macros

The inability to store comments is a show stopper by itself. Unlike a lot of high level code, assembly source code is usually festooned with comments. Back in the day, tutorials sometimes recommended to comment every line and some people took them at their word.

Macros are an immensely powerful tool, allowing the programmer to have some level of abstraction. For example, I expect every 6502 coder has their favourite macro for doing 16 bit addition. Unfortunately, the nature of their implementation makes it impossible to tell if a fragment of assembler resulted from a macro or not. I think you could live without macros, but they sure make life easier.

So, I think people would just put up with the slow assembly times resulting from assembling to disk. But, you should be aware that, back in the day, compiling or assembling anything was slow compared to modern times.

Answer 4

1. Assembly symbols are highly ambiguous (there is no 1:1 relationship between values and symbolic names). You seem to be assuming symbols can be addresses only - nope, they aren't. Imagine the following m68k code:

   address     equ    100
   offset      equ    100
   value       equ    100
   
               org    address
   start:      move.w #value,d0
               lea.l  address(a0),a0
               move.l d0,offset(a0)
               jsr    address+$100
               move   #'d',d0
   
   data:       dc.w   100
               dc.l   100
               dc.b   'd'
   
   
               dc.l   *-start         ; value "100" is completely lost
   

   The value 100 has (at least) 5 different meanings in your symbol table (addresses, immediates, offsets, relative or absolute, characters, bytes, words, longs), and in some examples (*-start) it doesn't even show up as a literal in the binary - when translating back and forth, you have the same problem a disassembler has when trying to make sense of symbols (when encountering "100" in the disassembly, you simply don't know which of the meanings/symbols to use), or you lose valuable information your original source code had. (Or you store that information someplace else and lose large parts of your memory savings)

2. You will be losing any comments and special code formattings like indentation and extra spacing if not stored in a separate place (without any memory saving). These are highly valuable in assembly code

3. The fact that code that has been generated from macros will not be visible. You will be losing that information if not stored elsewhere (again, no saving of memory)

4. You'd be losing all information on assembly pseudo-instructions (ORG, OFFSET, DC.B, the like).

5. You'd also be losing track of what's code and what's data, which is a significant problem when your disassembler would need to figure out whether 0C9H should be RET or DC.B 201 (a Z80 example). Modern disassemblers try to follow code streams to decide whether areas are code or data, but that's way beyond the capabilities of an 8-bit micro (and still not 100% accurate when jump tables are involved).

6. It's a huge difference whether a table was originally created as bytes, words or longwords, and then whether it was created as absolute or relative values (*-label or label). All of that is lost in a disassembly.

To an assembly programmer, it is a huge difference whether he works on a disassembly of code (what your proposal effectively implies) or whether he works with original code produced by another programmer. Depending on the complexity of the CPU and the original program, working on a disassembly is about a magnitude more difficult, in my experience. About 90% of the information what a program does is not in the binary. Even very sophisticated modern disassemblers/decompilers based on megabytes of code like IDA pro cannot re-create this information fully from a binary.

I think it's technically possible what you propose, but would be hampering productivity (a lot), or, if amended with extra information, will not produce the memory savings you seem to assume.

You could possibly amend lots of these shortcomings by storing the assembly program in a sort of tokenized form, but then it's no longer your proposal.

Answer 5

So my question is: did any assemblers with the above design, ever exist?

With the quite fuzzy question the answer is of course yes - and at the same time no.

There have been monitor programs that allowed entering and editing of object code in a source like way. Lines (instructions) could be entered as text (LDA #$3), symbolic labels could be defined (L1:) and used (JMP L1; LDA >L1) and so on. Lines could be deleted or inserted, all labels and branches within the code would be adjusted when doing so (*1). Some even allowed the use of symbolic constants, but only during entry.

Which makes them short of fulfilling all the above requests. As noted by RossRidge and others, keeping track of symbolic constants need to store each pointer name occurrence, not just the value. It's the same as for instructions, but there are fixed and a few - at least when programs get large, which is, as I understand the base for this idea. isn't it? (*2)

Another, related solution would be an intermediate way of storage used by such assembler-ish environment. Source is parsed when loading (or entering) into a tokenized form where each instruction, label and parameter is a single byte and optional pointers into symbol tables. A bit like most BASIC do. I think I remember a somewhat similar construction for the C64 from a magazin (*3) just I can't remember which and my google-fu failed me.

Then again, as soon as we include comments, the savings may become marginal (as pointed out by tofro and others). It's easy to see that comments can go past 50% of the source text. So even saving 2/3rd by encoding the instruction source, this will be only 1/3rd (or less) over all (*4).

All of this works of course only by making the assembler - or better IDE - able to handle it - vulgo BIGGER - leaving even less space to hold the source in memory. Maybe even so little that there's no space for the object code either. Keep in mind, the marvels of modern IDE are hundrets of megabyte monsters, only able perform with huge memory footprints and even larger disk space assigned on fast drives. Memories that large, that even the largest sources are comparable small and can be kept memory resident.

On an 8 bit machine this ends in endless overlay swapping, even if the source/object is kept in (remaining) memory. If you want the feeling, switch on UCSD Pascal on an Apple II and switch between editor, compiler and debug :))

Bottom Line: On classic machines the classic workflow is the only way to go when writing any reasonable sized program. Load editor, edit text, load assembler, assemble, load linker, link, load debugger, debug.

---

*1 - Heck, there where even hex-keypad versions working like this - ofc, instructions had to be entered as hex. IIRC the (extended?) monitor for the Junior Computer worked like that.

*2 - beside the assumption of interactive program development, a rather odd idea if I may add.

*3 - Being a BASIC program itself, AFAIR, memory was a precious.

*4 - Maybe a tad more if we compress comments as well, like using a 6 bit code :))