23

The BASIC used in the early ZX Spectrum (and also its ZX predecessors) had this weird thing where every single BASIC token was printed onto the keyboard, and for example PRINT or LOAD were entered by a single keypress. In particular the ZX Spectrum had so many tokens on each key, meaning a great deal of shifting and entering various keyboard modes to enter some of the less common tokens such as ATAN or READ.

I am interested to see how and why this was easier to implement than putting a simple tokeniser and string comparer in the ROM like everyone else!

7
  • 1
    Related, possibly useful - retrocomputing.stackexchange.com/questions/1734/…
    – Bob Tway
    Jan 26, 2018 at 12:52
  • 1
    I once actually wrote a ZX Spectrum tokeniser, so it would have been technically feasible, although I don't know whether there would have been enough room in the ROM for it. (And of course not tokenising allowed you to obfuscate your code in funny ways such as LET PI = 3.)
    – Neil
    Jan 27, 2018 at 12:08
  • 1
    I worked with a guy who had a Spectrum at home. After getting used to a more normal dialect of Basic at work, he found himself typing PRINTRINT etc at home.
    – badjohn
    Jan 27, 2018 at 13:21
  • @badjohn Could You tell, what the year it was?
    – Marecky
    Aug 9, 2018 at 7:20
  • @Marecky I think that it was 1984.
    – badjohn
    Aug 9, 2018 at 15:57

6 Answers 6

38

Contrary to other answers, obliging the user to enter BASIC tokens directly doesn't really save meaningful amounts of RAM. Many of its contemporaries such as the BBC Micro had BASICs where you typed keywords in full which were then immediately tokenised when you pressed enter. If anything, tokenised Sinclair BASIC generally had longer byte sequences than other BASICs due to parsing number constants and inlining the floating-point representation for performance. The latter is the source of bizarre memory-saving hacks on the ZX81 where e.g. PI-PI (3 bytes: PI, - and PI) would be used in preference to 0 (7 bytes: 0, 126 to indicate a floating-point constant, then the 40-bit constant).

Instead, it serves to save on ROM space. Remember that ZX BASIC started on the ZX80 which had just 4kiB of ROM for everything: Z80 control, I/O handling, character generation, and BASIC. Eliminating a tokeniser saved perhaps 500 bytes or so, which makes a real difference in such a small system. With the ZX Spectrum and its 16kiB ROM they would have had the option of supporting entering keywords in full, but chose not to. Partly perhaps because those 500 bytes could still do something more useful elsewhere, but most likely because the unusual input method was now a defining characteristic of ZX BASIC.

Another important reason would be marketing-related. Uncle Clive was very much a showman, and what better way for a machine to advertise what it was capable of than to label the keys with a treasure map of keywords begging to be explored?

4
  • 8
    I just wanted to say that exactly that keyboard layout hooked me on computing when I was a child, thanks to the rubber-keyed ZX Spectrum. Who needs a manual when the keyboard itself shows you what you can do?
    – Paul Smith
    Jan 26, 2018 at 15:32
  • 12
    Actually we used to use NOT PI for 0 (and there was also SGN PI for 1 and INT PI for 3 of course).
    – Neil
    Jan 27, 2018 at 12:03
  • Good answer, especially the final paragraph. From memory, the 16K ZX Spectrum ROM didn't have much spare space, so even if it was just 500 bytes, something else would have to be sacrifised. I think you're right about the labels on the keys though, definitely a genius marketing move. Jun 3, 2018 at 14:26
  • The spare rom space as I recall was meant to contain the Interface One code or part of it.
    – AndyF
    Jan 13, 2020 at 22:44
20

A purely speculative answer: it's a user experience improvement.

The ZX80 is unable both to process keyboard input and to maintain a stable display. It has the Z80 itself step through display bytes and programmatically generate vertical syncs, which requires it to be in a loop that is tightly synchronised to the video signal. It's able to do a fixed-length test for whether a key is pressed within that loop, but as soon as it needs to branch out and process a key press, it enters variable-length code, and loses synchronisation. The user observes this as a little screen roll.

The keyboard itself is also pretty terrible: a flat membrane, no tactile feedback. Like a smartphone except that your eyes aren't anywhere near it.

Therefore minimising keystrokes means: less typing, less screen rolling.

The ZX81 has the same kind of keyboard and still rolls in fast mode, and the ZX81 ROM is backwards compatible, being installable in a ZX80 (though slow mode, which produces a steady picture even when typing, won't be available because it relies on an extra interrupt generator). The ZX Spectrum's ROM is a direct descendant of the previous two and, anyway, single-press entry was by then associated with the brand.

EDIT: it's from an emulator I wrote rather than a real machine, but it's getting this result by really emulating a CRT, not because I think it's cool, so I maintain that this is an accurate representation of the real ZX80 typing experience:

screen roll artifact

9
  • Ah, so it's an inheritance from earlier designs, where it's definitely necessary. Jan 26, 2018 at 16:46
  • As a former PC/GWBASIC user that hated ZX Spectrum BASIC because of that damn keys with tokens, I might tell you user experience improvement might not got anything to do with that. Jan 26, 2018 at 20:29
  • 3
    @Rui The tokens are not so useful on the ZX Spectrum's keyboard on which you can touch type, but on the ZX80/ZX81's tiny membrane keyboard, it was nice to be able to save keystrokes. (But not so nice having to hunt for the right token!) Jan 26, 2018 at 20:45
  • 1
    @traal Granted....BetaBasic? and MegaBasic had as their main attractive being able to touch type at will. Jan 26, 2018 at 21:22
  • 1
    BetaBASIC was the model for the Sam Coupe's BASIC, which inherits much the same interface as the 48kb Spectrum and its predecessors in a scrollable listing and a line editor at the bottom but does away with the single-key token entry in favour of ordinary typing. Even by its release in 1989 that felt very old-fashioned to me, even just as compared to my brother's 128kb Spectrum. I used it a lot though; it definitely felt much easier to me than a 48kb Spectrum.
    – Tommy
    Jan 26, 2018 at 22:30
7

I wasn't necessarily easier - it was down to money.

Remember these were the days when the single most expensive component of a computer was its memory. For many micros of the time, a simple memory expansion kit would cost more than the base model machine itself. Sinclair was very clever at squeezing a working system into the smallest possible memory footprint.

Adding a tokeniser to read multiple keystrokes would have required extra ROM space for the software and extra RAM space to hold the keystrokes while a command was being built up. By using this form of key entry, the ZX80 managed to ship as a working machine with 1kb of RAM. The ZX81 and Spectrum followed on from this, deliberately undercutting the competition on price by minimising the use of expensive memory chips.

By the time the Sinclair QL came along, memory had reduced in price and so the drive to be so excessively frugal had reduced. Sinclair continued to undercut the competition by saving in other ways - for example Phloopy tapes instead of floppy disks.

7
  • 3
    The ZX Spectrum has a keyboard buffer which is easily large enough for a reasonable line of BASIC. And I'm not so convinced that a tokeniser will take up much more space than the tables and tables necessary for the 5+ tokens per key! If you could elaborate on that, however... Jan 26, 2018 at 10:54
  • 1
    You can put the tables in ROM, which was much cheaper. Also, you might not need them - if the bytecode the interpreter uses matches the format of the keyboard, you don't necessarily have to translate anything. Jan 26, 2018 at 11:04
  • @fluffysheap Of course the token table is already in the ROM (occupying 368 bytes): skoolkid.github.io/rom/asm/0095.html It is necessary for showing the tokens in the text representation of a program. Jan 26, 2018 at 11:55
  • 1
    @Wilson The keyboard mapping is in the Key table: skoolkid.github.io/rom/asm/0205.html --- Here is the main URL for the ROM disassembly: skoolkid.github.io/rom --- another disassembly in PDF: esocop.org/docs/CompleteSpectrumROMDisassemblyThe.pdf Jan 26, 2018 at 12:24
  • 2
    It's worth noting that there are replacement ROMs available for the Spectrum which happily ignore keyword entry - see e.g. SE Basic - and these modern ROMs have been written with the constraints of trying to be compatible with the entry points into the original ROM. Jan 26, 2018 at 14:21
2

I am in doubt it was due to a special technical reason. Finally I can not judge really but I am convinced there are a couple of advantages of this design.

My first computer/programming experience was with ZX Spectrum+. In contrast to its predecessors the keyboard of this model was outstanding till today. It was very well designed and very responsive.

For me to have all the available commands printed on the keyboard helped very much to make first steps in programming at all. Further you have been forced to enter syntax correct code. The editor by default only accepted correct commands syntactically. The system was easy to learn and not very complicated at all. I liked it very much.

enter image description here

4
  • 3
    That's well and good, but how does it answer the question, "why was it easier to implement?" Jan 26, 2018 at 14:45
  • 2
    My intention was to attract the matter of fact that such input system was not necessarily related to technical aspects only.
    – Thomas
    Jan 26, 2018 at 14:47
  • 1
    This could be reworded slightly to be more about implementation (e.g. less syntax checking (as input validation) had to be done) and so to actually answer the question. I'll mark it as "Looks OK" in the Low Quality Posts queue but I'll be coming back to it.
    – wizzwizz4
    Jan 26, 2018 at 22:14
  • In my experience, it was seen as an improvement by some users, and therefore it was a selling argument. I remember kids having an Spectrum bragging about this feature to kids having a Commodore. The point was that it was easier to program an Spectrum because commands were written in the keyboard.
    – Pere
    Jan 28, 2018 at 21:02
2

The 128K model (and its Amstrad made successors) did use a tokenizer in 128 Basic mode.

In this mode, you had to type in the command lines in full text, then syntax checking was made upon line validation rather than on the fly as with 48 Basic, and if correct, then the line was tokenized, then either stored (if preceded by a line number) or executed.

So there actually WAS a tokenizer in Sinclair Basic. In 128K mode, involving an additional ROM bank.

Anyway, it meant an additonal step had to be added to analyze the syntax and translate keywords into tokens.

The token part itslef was necessary anyway, for several reasons : 1) shorten up the code ; 2) accelerate programs execution, as the necessary translation part was done at typing time, not execution time ; 3) since a lookup table is always needed to associate each machine code subroutine to each keyword, matching them at typing time once and for all is another way to accelerate programs execution.

So, yes, tokenization is always a good idea. BTW, whan you wanted to add new keywords, you just had to trigger an error that led to running your own code, using the existing "hook codes" mechanism originally intended for the Interface 1 additional keywords (that were printed on the keyboard, but not actually present in ROM).

There were two common ways to do so : either alter some of the existing commands syntax, or adding new parameters or new peripheral types (such as MGT disc interfaces), or leading new keywords by a character that could be typed in "K" cursor mode, for instance any symbol-shifted one (an "!" was usually used a header character).

The point was to trigger a syntax checking error, trap it and redirect the command to your own code, so it would have the system to accept the line when typing it, and run your own subroutine at runtime.

Of course new commands this style weren't tokenized, so they occupied more space and had to be reinterpreted at runtime, thus being slower than built-in ones (though since this was all done in machine code, I doubt you could notice anything).

4
  • 1
    There is one caveat here because the 128K introduced the SPECTRUM and PLAY commands - the first to go into 48K basic, the second to use the AY chip. The tokens used were the T and U UDGs and hence there was incompatibiility with BASIC programs that used those characters.
    – cyborg
    Apr 23, 2019 at 19:53
  • Some Basic programs are slower or certainly feel more sluggish in 128K mode.
    – AndyF
    Jan 13, 2020 at 22:46
  • They shouldn't be, though. The contended memory banks are the same, well with the exception of the +2 (black and later) and +3 models, because Amstrad used the wrong address line to select each 64k memory banks).
    – Z80Man
    Sep 6, 2022 at 2:00
  • Aaah... I can't insert a line feed, the Enter key posts the comment ! So, if it's an original 128k or grey +2 model, then only the lower 16k bank is contended and there should be nothing to slow down BASIC programs. Of course, it's different for machine code, because it could be running in a contended RAM bank mapped in the top 16 area. But it should be hard to meet the problem in pure BASIC, because bank switching would be dangerous with the system stack on top of the BASIC program area... But timing problems with the +2/+3 games occured because of the different contended RAM banks.
    – Z80Man
    Sep 6, 2022 at 2:16
0

My bet is because of small amount of RAM in BASIC. You need to take in mind there is the framebuffer, heap/variables and stack. If the code where represented as ASCII text it would be considerably longer. I often hit the memory limit in BASIC while writing small games in my early days of programming. For example this (first BASIC program I found in my archive):

 7901 BORDER 7: POKE 23693,56: CLS 
 7902 LET f=1: LET n= PEEK 23635+256* PEEK 23636: LET vars= PEEK 23627+256* PEEK 23628
 7903 LET k=16383: INPUT CHR$ 8;" LINE start: ";od;" koniec: ";do; AT 0,0;"Hlada sa: ";a$: CLS 
 7904 LET g=0: IF od <> ABS od OR do <> ABS do THEN LET g=1: LET od= ABS od: LET do= ABS do
 7905 IF n >=vars THEN STOP 
 7906 GOSUB 7917: LET n=n+1: GOSUB 7918: IF l<od THEN LET n=n+b+1: GOTO 7905
 7907 IF l>7900 AND l<7930 THEN LET n=n+b+1: GOTO 7907
 7908 LET t=n: LET n=n+1: IF l>do THEN STOP : STOP : STOP 
 7909 LET m= PEEK n
 7910 IF m= CODE a$ THEN GOSUB 7919
 7911 IF m= CODE """" THEN LET f=1-f
 7912 IF f AND (m= CODE ":" OR m= CODE " THEN ") THEN LET s=s+1: LET t=n
 7913 IF m=14 THEN LET n=n+5
 7914 LET n=n+1
 7915 IF m=13 THEN GOTO 7905
 7916 GOTO 7909
 7917 LET s=1: LET l= PEEK (n+1)+256* PEEK n: LET n=n+1: RETURN 
 7918 LET b= PEEK n+256* PEEK (n+1): LET n=n+1: RETURN 
 7919 PRINT PAPER 5;l;",";s;: PRINT TAB 8;: LET f=1: LET n=t
 7920 LET n=n+1: LET m= PEEK n
 7921 IF m= CODE """" THEN LET f=1-f

is 1058 bytes as ASCII and 867 bytes as a BASIC tokenized program file (including header). If I do the same for simple BASIC game like Zednik ma zizen (not mine but very popular back in the days) then:

ASCII:   45619 Byte
.P file: 40995 Byte

The 5KB difference might not look like much but at the time it was. This game stretches the limits as there is only 42240 bytes available, minus the BASIC variables, and as ASCII text would not fit just the code alone ...

When the memory size changed (like on ZX128) they added text representation too ... (that is why D40/D80 FDC+FDD for ZX48K does not work in such environment with BASIC as it use tokens too...)

D40

Another reason is speed. As ZX has BASIC interpreter processing text is much slower than processing tokens. For compilers it does not matter much as after binary is created no further processing occur during runtime. But interpreters must process each line of code when it executes. So loops and GOTO/GOSUB invokes processing usually already processed lines again and again ...

Yes it is possible to use text editor and token representation together but you must realize that there where not much tools at that time and code that and properly debug it would probably take much longer than they wanted to spend on it. Also conversion back and forward between both representations take time and coding could get sluggish ...

4
  • What is D40/D80? Jan 26, 2018 at 11:17
  • 1
    @Wilson added link and image
    – Spektre
    Jan 26, 2018 at 11:47
  • 5
    You can still tokenize when hitting ENTER. Most BASICs of the time did this. Also note that the representation of number literals in Spectrum BASIC programs is actually longer than just the ASCII representation (it includes the binary floating-point value as well as decimal) and this is done for speed reasons. Jan 26, 2018 at 14:06
  • 3
    The bold is really not adding any useful emphasis here.
    – Joe
    Jan 26, 2018 at 17:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .