That encoding is not just the token table but as well the BASIC charset and the screen charset which offers each of the 64 printable characters as Black-on-White (00h..3Fh) or White on Black (80..BFh). Much like the Commodore PET or the Apple II does.
One is that it's not ASCII.
Who needs ASCII anyway?
Neither Commodore nor Apple did go all out ASCII with their encoding.
Some parts are in the same order,
Which are for things that commonly need to be ordered by increasing representation, like
- Numbers (0..9),
- Letters (A..Z) and
- Graphic symbols (bit wise encoding to allow SET/RESET operations)
Numbers and Punctuations are both lower in value than Letters, much like with ASCII. In fact, the ZX81 yields an advantage over ASCII by having all 4 types in distinct and closed groups, not mixing Punctuation at random places around Numbers and Letters
so it would seem conversion is non-trivial.
Since the machine doesn't operate in ASCII no need for conversion exists:
CHR$ 38 yields
CODE "A" gives
The real question though, is why does it have all the letters, digits and punctuation in the table twice?
Its the same reason why the Apple II or the PET has them as well each twice (*1): To display each in two versions: Normal or Invers
For the ZX81 token list and character set and display character set is put into the same code space.
The character set consists of only 32 character existing in two versions:
- 000..063 Black on White
- 128..191 White on Black
This is way better to be seen when looking at an ordered image like this
(Image taken from Wikipedia)
Or more orderly shown in a nice table at the ZX81 Character Set Wiki page.
What is the purpose of this?
Simplifying the whole character and screen handling.
As already mentioned, Apple II and Commodore PET feature similar duplication within their screen character set. Those machines must parse every output according to a ASCII to Screen Code translation while observing modal settings. In case of the Apple two the screen character subset is selected according to a general setting made by the BASIC commands
BLINK, while on the PET the string to be printed has additionally to be parse for the control codes of
RVS OFF, each changing the character translation for any subsequent character PRINTed.
The ZX81 does not have to flip encoding according to some mode, nor does it need to provide additional BASIC commands, like the Apple, or parse output, like the Commodore, to handle modal setting. A character within the Display File (*2) will simply be output the way it is (*3)
One More Thing:
Looking at above Wiki table might leave one wondering what the codes in row 7x are, as the 'printout' image shows them as question marks.
They are, with partitial exception of New Line (76h), and regarding string output exactly what their top bits suggest: unused tokens. These codes are only produced and relevant due keyboard input. They control the editor but may as well be inputted (read) by a BASIC program using
Within the Display File New Line is used to mark the end of a line, filling the actual screen line after it with nothingness.
*1 - Well the Apple II even 4 times, including blinking and a redundant white on black copy.
*2 - Display File or D_FILE is what Sinclair called the structure holding the picture to be shown. In many way similar to a screen buffer, in others, as seen quite different.
*3 - And the (PRINT) routine inserting a character into the display file does only have to check against a a fixed bit mask to differentiate between characters and tokens. Anything with
x0xx.xxxx ist a character to be copied, while all other are potential tokens and decoded into their representational strings (Like 0DEh into
THEN) accordingly (*4), or if not found into a question mark.
*4 - Which in turn simplifies the
LIST command a lot, as most tokens only need to be PRINTed without any further evaluation.