7

I only learned of BASIC09 today, and I'm trying to wrap my head around it.

The manual talks about it being a compiler, but I'm not sure I see significant differences between its I-code and tokenized BASICs in general. I know you can PACK it, but that seems more like a symbol stripper.

I am also curious if anyone out there has a working version of BASIC09, preferably on the CoCo3? I would love to run the Ahl and Byte benchmarks through it.

3
  • Benchmarks: I don't know of any for the 6809. Here typewritten.org/Articles/Benchmarks/primes.html you have some for the 68k version (filter for "68000"). That shows it's outperforming MS Basic on a comparable platform nearly by a factor of 4 on a prime benchmark. Here vintageisthenewold.com/want-fast-basic-try-basic09 is a comparison with ColorBasic on the CoCo - factor 15 slower. – tofro May 14 '19 at 12:04
  • It's a huge difference between tokenized BASIC, which is still the same source code and bytecode, where actions according to data and context get coded. For example adding two integers result in a direct call to an integer add instead of calling a generic adding routine with all the type conversion overhead. Similar for variable access and so on. – Raffzahn May 14 '19 at 15:38
  • 1
    I'm pretty sure that the standard NitrOS-9 distro includes Basic09. You shouldn't need to go searching for it separately. – Brian H May 14 '19 at 15:49
6

I can't speak to BASIC09 specifically, but there's quite a difference between an "i-code" and a tokenized form.

At a minimum, the i-code, in the end, need not look at all like the source code, whereas the tokenized form effectively IS the source code.

Many BASICs, when you type LIST, simply parrot back out the tokenized data in long form. With i-code, you can't do that.

Another example is if you look at the DEC BASICs. They used "statement modifiers".

Consider:

10 SUM10 = SUM10 + A(I) IF A(I) > 10 FOR I = 1 TO 10

While this could be represented in tokenized form (in order to regenerate the source line), it can not be easily interpreted executed from that tokenized form.

Token based runtimes, much like i-code, simply take what they see and execute it in the context their in. But you can easily see the runtime encountering the SUM10 = SUM10 + A(I) with out knowing that it should do so yet (due to the IF statement) or even that I is properly in range or not (since it's not even been initialized properly yet).

Sure, you could jump through hoops. Put in "invisible tokens" or whatever specifically to support this kind of construct, but in the "every byte counts" of vintage BASICs, that's a lot of work for not a lot of value. But since the DEC BASICs WERE compiled, rather than just interpreted raw, the system was able to take statements like this and rewrite them internally to be more "normal".

Also, consider:

10 A = 1 + 2 * 3 / 2 + 4 * SQR(2)

In a normal tokenized system, the expression runtime has to account for the operator precedence AT runtime to properly evaluate this expression. Having to do the multiplication and such before the additions, etc.

Whereas in a compiled form, it can be reduced to an RPN style where the evaluator doesn't have to worry about that any more. This makes the runtime faster, since it's much simpler.

1 2 3 * 2 / 4 SQR(2) * + +
1
  • "Many BASICs, when you type LIST, simply parrot back out the tokenized data in long form. With i-code, you can't do that." - actually, it did just that. There turned out to be a discussion in the printed version of the manual that talks about this in more detail. The big difference is that variables were not in lookup tables but replaced by direct memory references, and likewise pointers to lines and similar were replaced by offsets. – Maury Markowitz May 15 '19 at 0:05
4

The simplest way to get going with BASIC09 would be to get the NitrOS-9 "ease of use" edition. Then if you have a CoCo 3 with 512K RAM and a CoCo SDC (which lets you keep disk images on an SD card), or have MAME or the VCC emulator set up to emulate a CoCo 3. (On the off chance that you have a Matchbox CoCo, an FPGA-based system that Roger Taylor has created, there's a version for it, too, thanks to the work of Bill Nobel, but for now, the CoCo 3 with CoCo SDCand those two emulators are the supported setups.)

About BASIC09 vs. tokenized BASICs of that era (almost all Microsoft-based, ultimately going back to Altair BASIC): the extreme memory constraints of Altair BASIC (by default the Altair came with 1K of RAM; the first Altair BASIC version needed 4K) forced some design decisions that MS stayed with for a long time, at a catastrophic cost in runtime. There's a series of books, starting with Color BASIC Unravelled, that have a disassembly of the various versions of BASIC that run on the CoCo. (Also, MS has just now opened the source of GW-BASIC, which I think still has the same basic, you should pardon the expression, structure.)

The tokenized MS BASICs of the time make almost no changes to the source code as you type it or load it; lines are put in a singly-linked list with the line numbers stored as two-byte integers. Language keywords are converted to one-byte codes and built-in function names to two-byte codes, but that's it. As lines are interpreted, variable names are looked up by linear search through a symbol table that only keeps the first two characters of variable names (and new entries made when the lookup fails). Every single time a line is interpreted, constants are reconverted from text form to internal format, expressions are reparsed for evaluation. Every GOTO or GOSUB forces a linear search of the statements (though if the number is greater than that of the current line, at least it doesn't have to start from the beginning).

All this means that tokenized BASIC programs of any size fall victim to coding practices that try to reduce the enormous overhead of interpretation at an dire cost in legibility and maintainability. Look at any Color BASIC listing in any Color Computer magazine and you'll see the results:

  • statements crammed together to fit as many as possible on each line
  • gratuitous references to frequently-used variables at the beginning of the program to force them to appear earlier in the symbol table
  • assigning constants to variables so that they need only be looked up, not reconverted
  • taking advantage of Color BASIC's highly ad hoc parser to omit as many spaces as possible from the source code
  • typing constants in hexadecimal where possible for faster conversion (just imagine F = $20 + $9*C/$5--hey, that's legible, right?)

basic09 (BASIC09's text-oriented IDE, in today's jargon), on the other hand, parses code as you enter it, converting it to I-code. I-code is often likened to Pascal P-code or Java byte-code, but in one way it's quite different, because basic09 has to serve two masters: one is the programmer editing, listing, and debugging code, and the other is the programmer running code. If you have a separate compiler, your byte code can, for example, have simple conditional and unconditional branches like typical computer instruction sets. BASIC09 I-code, OTOH, has constructs that correspond to BASIC09 statements and control structures, so it can, by keeping around symbol table information that is not used at runtime, recreate a very close approximation of the code you enter or load. It's not exact, because

  • constants are converted to internal form, so they may not be exactly what you typed
  • expressions are converted to RPN, and the conversion back to infix form will not include any unnecessary parentheses you may have typed
  • variable, parameter, and procedure names are only stored once in a symbol table, and symbol table lookups are case-independent. If you first type camelCase, and later type CaMeLcAsE or camelcase (in the same procedure), they will all appear as "camelCase" in the listing; all will refer to the same variable or parameter. (That gets rid of the main objection to camelCase, namely having to hit the shift key over and over.)
  • The listing is prettyprinted, with standard formatting and indentation--we're talking almost three decades before Go's designers created gofmt to get rid of all the bikeshedding and arguments about layout. Color BASIC rewards illegibility; BASIC09 makes your code legible. Conversion of source to I-code, along with BASIC09's broader selection of types, means that a BASIC09 program for a given task will almost certainly run much faster than a Color BASIC program for the same task. (I've only seen one exception, which was a very special case for a small benchmark; see "Filling the Screen with Text".)

You'll notice I said "procedures". Tokenized BASIC programs of that era are monoliths, "big balls of mud" as Foote and Yoder put it. All variables are global (and with only the first two characters significant in variable names, it's very hard not to reuse the same name by mistake). A BASIC09 program, OTOH, is a set of procedures (which can be machine language or compiled code, as long as it follows BASIC09's calling conventions). BASIC09 procedures have local variables and support recursion; breaking a problem down properly can thus not only make code easier to read, but make it use less memory--important when a process can't get to more than 64K of RAM.

I hope this answers your question and also encourages you to check out BASIC09.

2

The manual talks about it being a compiler, but I'm not sure I see significant differences between its I-code and tokenized BASICs in general.

Well, so called 'tokenized' BASIC isn't really tokenized, but rather put in a shorthand. Tokenization does require to put everything into token format, which isn't true for average BASICs. The term 'crunched' used by Microsoft is far more appropriated here (*1).

BASIC09 did not only turn recognized keywords into single byte identifiers, keeping the rest (constants/variable names) as is, but puts everything into tokens, which is a virtual machine instruction (*2) as well.

A variable reference is a token of its type (byte/integer/float/string/complex), followed by the address of its value. Similar constants, but with the value following. So unlike 'standard' basics, the interpreter doesn't has to figure out every time what it's about - no matter how lightweight - but simply executes the token.

So far this might look like some sophisticate cruncher, but all operators are as well available in typed version. For example there are

  • two different - instructions for byte/integer and float
  • three different + instructions for byte/integer, float and strings
  • four different = instructions for byte/integer, float, strings and bool
  • four different <> instructions for byte/integer, float, strings and bool

and so on. Depending on operand types used in an expression BASIC09 will compile the appropriate instruction. This extends to whole BASIC instructions aswell, as there are for example two NEXT instructions for the VM, one that accepts an integer coutner variable and one for float. The later ofc slower.

Next, all expressions are turned into RPN. For example an IF statement will look like this:

(Made arbitrary complex on purpose)

IF MID$(Input$,Pos,1)="\" AND Found=0 THEN

is compiled to

  • <IF>
  • <String-Var> address of Input$
  • <Integer-Var> address of Pos
  • <Byte-Const> value 1
  • <MID$>
  • <String-Const> value "\"
  • <String Comparison Operator '='>
  • <Integer-Var> address of Found
  • <Byte-Const> value 0
  • <Integer/Byte Comparison Operator '='>
  • <OR>
  • <Invisible Goto> address of ELSE/ENDIF
  • <THEN>
  • <End of Line>

It's quite clear to see that a stack machine is operating along the instruction stream of an expression.

<End of Line> is rather syntactic sugar to insert automatic line breaks (and indentation) when listed, as BASIC09 prefers. Similar all expressions will be 'normalized' and alternate spellings like ><put into their normal form <>.

While I-Code structure is (naturally) close to BASIC, it does as well show differences. For example the TO and STEP parts of a FOR-NEXT are (together with the repeated variable reference) compiled to the NEXT instruction, saving the need to lookup what to examine for end condition and step width. In addition the SEP value will be always compiled as part of the NEXT instruction. As result the FOR statement will only be executed for initialisation. all following iterations will direct jump to the next instruction after FOR (*2).

And so on.

Long story short:

I-CODE is not simply crunched (tokenized) BASIC, but consists of instructions toward a virtual machine, which just happens to be very BASIC compatible.

I know you can PACK it, but that seems more like a symbol stripper.

Yes, it that's what PACK is for. It removes all remaining source reference, like variable names, to save space. In addition it more aggressive optimizes expressions than the interactive interpreter does. After packing a program can only be executed - usually by RUNB.

Keep in mind, OS/9 is a RAM based system and so is BASIC09. BASIC09 (the program) is loaded into memory, thus reducing the RAM available to the user program. RUNB is essentially just the virtual machine (and BASIC runtime). By replacing BASICC09 with the much smaller RUNB more RAM was available by essentially dropping the editor, compiler and debugger parts (*3). IIRC BASIC09 was more than 20 KiB, while RUNB was about half of that. so 10+ KiB more space for a user program is quite a lot.


*1 - Yes, BASIC's come in many colours, including some that normalize way more than MS did.

*2 - This is not done for WHILE loops, as they are rejecting structures, unlike FOR which iterates at least once.

*3 - Well, and even more after packing.

6
  • Tokenized is the more widely-understood term. ‘Crunched’, to me at least, either means the no-spaces style (FORX=1TO10) allowed by earlier MS BASICs and BBC BASIC, or join lines, shorten variable names, etc – scruss May 24 '20 at 18:35
  • @scruss True, except tokenized is a well understood term when it's about languages, and it's usage with MS-BASIC (and most others) is simply wrong. BASIC09' I-Code is an exception from this rule. Also, Mr.Gates himself called it crunching :)) – Raffzahn May 24 '20 at 19:38
  • “UPN” usage above - Is that like RPN written uʍop ǝpᴉsdn? – scruss May 25 '20 at 13:09
  • @scruss Drats. Nop, it's just the German term Umgekehrte Polnische Notation :) – Raffzahn May 25 '20 at 13:33
  • 1
    I see. Poor old Łukasiewicz, his name isn't that hard to spell or pronounce … – scruss May 25 '20 at 14:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.