Why the disparity between the screencodes and the character codes?

Question

The Commodore 64, and possibly others, I don't know, have character codes (These map integers to characters, in the same way that ASCII does) and there are screen codes (The offset which the VIC-II will use to look up the bitmap in the character set).

The character codes are used in file names, I guess BASIC tokens etc also. The screen codes are only used when putting some letter on the screen! Then there are routines to convert between them. Presumably, the ROM also needs to do exactly this thing.

So is there a reason why not put the bitmaps in the character ROM in the same order that the characters are laid out by PETSCII? That would save people from having to do this daft, daft conversion.

Answer 1

For the C64, there's a simple reason (probably among others): Control characters vs. reverse video.

There are two character sets of 256 characters and two fonts of 256 glyphs. Both have the following in common:

• The font has 128 glyphs twice, in normal and reversed display
• The character set on the other hand has quite a few "control characters", like clear screen, issue newline, change color, and also enable/disable reverse video

Therefore, a 1:1 mapping simply isn't possible.

The system is however designed in a way enabling you to use your own font, and you can encode it however you like -- a while back, I created a font using the IBM CP-437 encoding, this is quite nice for e.g. on-screen docs.

Answer 2

Most microcomputers treat character codes and screen codes differently. This is because they serve different functions and arranging them both the same would compromise the functionality or implementation of one or the other.

Screen Codes

The function of a screen code is to encode, at a position in the frame buffer, what particular glyph should be shown by the video system at that position on the screen. Additionally the code may include information modifying the display of the glyph, such as a bit indicating whether the glyph should be displayed in positive (white on black background) or inverted (black on white background) form.

The patterns to generate the glyphs are usually stored in ROM (though sometimes RAM) and looked up by address. If the screen code directly encodes the (usually low portion of) the address in some of its bits this makes simpler the hardware doing the lookup of the glyph from the screen code. This is desirable to reduce both the hardware cost and the complexity of the system. (The reason cost reduction is desirable is obvious. Reducing complexity reduces the time spent on design and debugging and also reduces the chance that a bug will go out in the final product.)

This tends to make screen codes map fairly closely to the hardware used to generate the glyphs. For example, if there are 64 glyphs available in the system (typically numbers, punctuation and upper-case letters), they're likely to be stored at addresses whose lowest six bits are hexadecimal $00 through $3F (binary xx000000 through xx111111).

You'll note a difference if you compare the above to the ASCII character set encoding, where this range, with all the bits above the lowest six set the same, covers either:

• the (non-printing) control characters, punctuation and numbers ($00-$3F), with no letters at all, or
• the upper case and lower case letters (and a bit of different punctuation), with no numbers at all.

Character Codes

Character codes have two characteristics that are generally different from convenient screen code numberings as described above.

First, they almost invariably have non-printing control characters which, instead of requesting that a glyph be printed, request some other action on the part of the printing device. For example, in ASCII the LF ($0A) character generally indicates to a printer or screen that it should print subsequent characters on the next line.¹ Control characters are also often used as instructions to code that is processing characters; ASCII NUL ($00) is often used to signal the end of a string, for example.

Control characters are unnecessary in screen codes, however, and a non-printing code simply wastes space in the list of screen codes.

Second, it's common to want to use a standard character set and encoding (e.g., ASCII or Unicode UTF-8) for compatibility and easy data interchange with other systems.

Using a standard fixes the codes you can use, and often in a way inconvenient for use as screen codes, as we saw with ASCII in the previous section.

Summary

So, for the reasons described above, generally screen codes are easier to do in one way, and character codes in another. Fortunately this is not difficult to handle: most microcomputers provide a small routine that will convert character codes to screen codes. A routine to "print a character" is generally wanted anyway, since it lets commonly used logic such as "print a character just after the last one printed on the screen" be handled in one place (which also avoids co-ordination problems between different routines that both print), so adding this conversion is usually a trivial cost.

---

¹_{This is a simplified example; what the control characters do is often far more complex and system-dependent than described here. But the general idea is the same.}

Answer 3

The Commodore 64, and possibly others, I don't know, have character codes (These map integers to characters, in the same way that ASCII does) and there are screen codes

To start with, this includes the Apple II - as well as several others.

So is there a reason why not put the bitmaps in the character ROM in the same order that the characters are laid out by PETSCII? That would save people from having to do this daft, daft conversion.

For one, isn't there an OS to do that job? (*1)

The main reason is simple: Rearranging Them Saves Hardware.

The other is that rearranging them enables additional features.

---

For example, the Apple II had only a ROM for 64 different glyphs. Arranging them in a way to suit ASCII encoding, would mean that ROM space for 128 glyphs would be needed - or additional hardware to do the transformation on the fly. That would have asked for a ROM chip double the size (*2) or some clever logic doing the transformation on the fly. Replacing a rather expensive chip with a few lines of machine code seems like a good trade-off. Especially one in line with Woz' constant struggle to cut down the chip count. Isn't it?

Similarly, Sinclair's ZX80/ZX81 character set follow the same reasoning of cutting down the needed ROM space. Except here, 11 punctuation characters were switched for graphic symbols to allow for 4 pixel per character cell graphics.

The Sinclair ZX Spectrum extends this to 128 codepoints to include full ASCII charset with lower case and all (*3) punctuation plus the drawing symbols.

On the PET the reasoning was a bit different. While it had a character ROM holding 256 glyphs, they were organized as two character sets of 128 each, so standard character handling could be kept as 7 bit. Further, the PETSCII encoding corresponded to what the keyboard delivered, and the ROM was organized accordingly.

So while it made sense for the first PET, the C64 and other 8-bit Commodores only inherited it for compatibility.

---

Bottom line, by separating the functionality of having ASCII on the software side, while having a different hardware representation on the CRT controller side, enables ways of optimization - much like any other abstract interface does.

---

*1 - NEVER. EVER. Assume that a certain binary representation of your data is the same over all steeps of processing or across different devices and networks. This and assuming that binary will be unharmed in transmission are eventually the two main reason for bugs ... well, after off-by-one that is.

• Never assume that binary representation of (character) data has a particular sequence or order.

• There are representations where he binary value of B is lower than the value for A, while C may be again lower, or higher or inbetween.

• Always keep in mind that recoding can and will happen.

• Never calculate between codesets. Use translation tables.

*2 - Or at least an additional half the size answering to the missing 31/32 glyphs twice.

*3 - Within reason.

Answer 4

20 a = peek(1024 + x + y * 40)
30 if a > 127 then c$ = chr$(18):rem revers on
40 c$ = c$ + chr$ (a + 64 - (32 and a) + (64 and a) - (160 and a))
50 print c$

So line 40 changes screen code to character code. See +64, then -32 & -160 [that = 128 + 32] = 0, or -128 but then it's to be printed with revers on.

Answer 5

It's probably just a case of two separate engineering teams solving a problem at different times or just not talking to each other.

Graphics guys worked out the easiest route for them, software kernal/basic team used their encoding - later on - someone merged the two with a lookup.

My guess is its just historical C64 wasn't going to change 'anything' for the sake of it - even tidiness - they probably only changed the software components they had to - for cost reasons (including dev and debugging)

Involve humans in a design over time and you get compromises - layers of compromises - the C64 was not the first computer these departments and groups inside CBM built so it gets some baggage.