What is the format of the syntax tables in Apple II Integer BASIC?

Question

In Paul Santa-Maria's disassembly of Integer BASIC (text version), you can see the syntax table split into two parts, SYNTABL and SYNTABL2. I've seen references to the tokens being stored in reverse order, and indeed, the characters within a token are in reverse order, and the tokens themselves are counted down from the ends of the tables.

What is the exact format of the syntax tables? There are a bunch of extra bytes littered around, I believe to store different versions of tokens (eg. PRINT, PRINT <string>, PRINT <expression>), and some of the tokens lack extra bytes: some of them use char+32 vs char-32 to signal the first character, but again, some don't. And some that do also have extra bytes. In addition, SYNTABL2 ends with some extra bytes (are they usually skipped?) and SYNTABL starts with some apparently unused tokens, then some extra bytes of its own.

Is there a clear description somewhere?

Context: I started out looking at the code because I'm trying to follow the exact rules for adding whitespace when rendering Integer BASIC listings to text. But now I'm intrigued.

Answer 1

There's an interesting thread here: http://forum.6502.org/viewtopic.php?t=2037

The syntax table is in reverse order, as you've already noticed. An underscore is stored at the end of the line by the line input routine.

There are 32 categories (well, that's what I'm going to call them). Each category contains one or more rules.

The syntax tokens are:

$00: return with syntax error
$01: return with no error
$02: parse a comment character (any non-control character, stops at the end of the line)
$03: parse a string literal character (stops at a closing quote)
$04-$1F: categories $04-$1F, next token is required
$20-$3F: categories $00-$1F, next token (and beyond) is optional
$40-$5F: categories $00-$1F, end of category
$60-$7D: categories $00-$1D, end of rule
$7E: branch "backward" 2 bytes
$7F: branch "backward" 1 byte
$80: parse letter A-Z, next token is required
$81: parse number 0-9, next token is required
$82-$BF: parse character "-_ -- next token is required
$C0: parse letter A-Z, end of rule
$C1: parse number 0-9, end of rule
$C2-$FF: parse character "-_ -- end of rule

The category table (SYNTABLNDX in the linked listing) is at $E197. To address 512 bytes with 8 bits, each category starts at an odd address. Each entry in the category table is:

DB (ADDRESS_OF_START_OF_CATEGORY-1)/2

Category $07 is a numeric expression. So HLIN $07 AT $07 , $67 means HLIN AT , and the $67 indicates the end of the HLIN rule.

Category $00 is an unsigned number (line number), so LIST $20 , $60 (at $EC61) means the LIST , with the comma and the second line number optional

Category $19 is a numberic variable name, so NEXT $39 , $7E means NEXT , , ... i.e. NEXT A or NEXT A,B or NEXT A,B,C etc.

Sometimes keywords are duplicated to force a mismatch (and thus force the end of a category). For example, RNDX (at $ED69) can never be matched since it comes after RND (at $ED75) in the numeric function category (at $ED79). (The first three letters would match RND first.) Thus, RNDX matches nothing and effectively marks the end of the category.

If you're curious about the THEN TO STEP etc. keywords at $EC17, see this thread:

viewtopic.php?t=726

If you have further questions, feel free to ask.