29

On the Commodore 64 and VIC-20, the LIST command will abort with a ?SYNTAX ERROR on any line that contains a REM statement with a Shift+L character. A minimal program triggering the error would look as follows:

10 rem L

Some programmers deliberately started or peppered their BASIC programs with such "killer REM" statements to hide the source code. (Of course, the protection this provided was only rudimentary, and could be easily bypassed.)

As far as I know, this quirk affects only the C64 and VIC-20; the C128, C16, Plus/4, and various PET models don't care about the argument of REM statements. What I don't understand is why this quirk exists and how it works. Is this a bug or a feature of CBM BASIC? If it's a feature, was it ever documented anywhere by Commodore, and why is it present only on the C64 and VIC-20? If it's a bug, what exactly causes the LIST command to choke on a shifted L in a REM statement?

  • It also affects the CBM 8032, all the way down to the PET 2001. – cup Mar 16 '18 at 18:07
  • @cup On my PET emulation running BASIC 4, it results in a listing in which the shift-l is replaced by CONCAT i.e. 10 REM CONCAT – JeremyP Nov 20 '18 at 15:22
15

What I don't understand is why this quirk exists and how it works.

As usual, lousy programming. It's a routine that exists in next to every Microsoft Basic, but often modified by the receiving company. It's used to list a line. On the 6502 version space was a premium, so they tried to cut down as much as possible in tests. And lets be serious, a 'real' terminal can not produce codes above 128 so there is no chance someone would enter a token code somewhere. And what computer engineer in his right mind would create a machine using other codes than plain ASCII?

Is this a bug or a feature of CBM BASIC?

It's a combination of several little bits of 'less than perfect' ending up creating a bug.

and why is it present only on the C64 and VIC-20?

Because here the token table is exact 127 bytes of token names plus one byte of zero.

If it's a bug, what exactly causes the LIST command to choke on a shifted L in a REM statement?

As described already, the list routine iterates over all program lines and in each it displays the line number and then parses all token/text and print it. Tokens are recognized by having the high bit set. Tokens on the C64 reach from $80 (END) to $CA (GO) plus $FF for PI. Whenever a token is detected, the token table is searched for (PI is handled separately) in the most primitive way: $7F is subtracted and then tokens are scanned and skipped with that value decremented until zero is reached. Then this word pointing at is printed. Not really fast, but who cares during listing.This all works fine and stable - even with unused tokens. Try enter a Shift + M, and you get a FOR ($81) listed.

Just $CB fails, as it points to the combination of a single zero and an 128 Byte overflow.

In conclusion it's a combination of a missing upper boundary check together with an addressing problem that only appears when using a character set not designed for and an input routine not catching it either.

The original routine can be found in all MS-Basic 1.0. For newer MS-Basics the token table is of a different length and the routine is modified - usually by using a 16 Bit Pointer instead of a fixed address plus offset - as their token tables are larger than 127 bytes.

  • I have seen this used in C64 Text Adventure games to hide the answers to riddles. – PhasedOut Nov 21 '18 at 19:35
  • There's still something unexplained: why do you need REM to trigger the bug? The same character in other places doesn't do the same thing. (And other graphic characters don't expand into tokens either) – Wumpus Q. Wumbley Nov 23 '18 at 0:37
  • @WumpusQ.Wumbley Because these ("at other places") characters are all part of strings, which are handled different (when ever a " is encountered, characters are direct outputed until the next "). – Raffzahn Nov 23 '18 at 0:53
26

Internally, a BASIC program isn't represented as the text you see when you list it, but as a tokenized data structure where each of the language keywords are represented in an optimized 1-character form. Basically, if the upper bit is set in a character byte (i.e. values >= 128/$80), it is processed as a token.

Note that this does not only apply to the shift-L case, fx entering "10 REM Shift + A" will list as "10 REM ATN". If you consult a list of the predefined BASIC tokens, you will find that 203/$CB is the last defined; this will match Shift + K. After this, you are presumably (I'm speculating, haven't actually pored over the ROM listing) referencing a non existing or otherwise erroneous token, and the interpreter bails.

That you can enter tokens directly would seem like a bug or at least design limitation that was found acceptable, but I've never seen any official stance on that. Earliest mention of this token quirk I could find was in a 1983 issue of Compute!.

  • The shift key sets the 0x80 bit in the character on this system? (As opposed to its usual function with alphabetics, of selecting upper case versus lower case) – another-dave Nov 20 '18 at 23:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.