Retrocomputing Stack Exchange is a question and answer site for vintage-computer hobbyists interested in restoring, preserving, and using the classic computer and gaming systems of yesteryear. It only takes a minute to sign up.
Sign up to join this community
I only learned of BASIC09 today, and I'm trying to wrap my head around it.
The manual talks about it being a compiler, but I'm not sure I see significant differences between its I-code and tokenized BASICs in general. I know you can PACK it, but that seems more like a symbol stripper.
I am also curious if anyone out there has a working version of BASIC09, preferably on the CoCo3? I would love to run the Ahl and Byte benchmarks through it.
I can't speak to BASIC09 specifically, but there's quite a difference between an "i-code" and a tokenized form.
At a minimum, the i-code, in the end, need not look at all like the source code, whereas the tokenized form effectively IS the source code.
Many BASICs, when you type LIST, simply parrot back out the tokenized data in long form. With i-code, you can't do that.
Another example is if you look at the DEC BASICs. They used "statement modifiers".
Consider:
10 SUM10 = SUM10 + A(I) IF A(I) > 10 FOR I = 1 TO 10
While this could be represented in tokenized form (in order to regenerate the source line), it can not be easily interpreted executed from that tokenized form.
Token based runtimes, much like i-code, simply take what they see and execute it in the context their in. But you can easily see the runtime encountering the SUM10 = SUM10 + A(I) with out knowing that it should do so yet (due to the IF statement) or even that I is properly in range or not (since it's not even been initialized properly yet).
Sure, you could jump through hoops. Put in "invisible tokens" or whatever specifically to support this kind of construct, but in the "every byte counts" of vintage BASICs, that's a lot of work for not a lot of value. But since the DEC BASICs WERE compiled, rather than just interpreted raw, the system was able to take statements like this and rewrite them internally to be more "normal".
Also, consider:
10 A = 1 + 2 * 3 / 2 + 4 * SQR(2)
In a normal tokenized system, the expression runtime has to account for the operator precedence AT runtime to properly evaluate this expression. Having to do the multiplication and such before the additions, etc.
Whereas in a compiled form, it can be reduced to an RPN style where the evaluator doesn't have to worry about that any more. This makes the runtime faster, since it's much simpler.
1 2 3 * 2 / 4 SQR(2) * + +
The simplest way to get going with BASIC09 would be to get the NitrOS-9 "ease of use" edition. Then if you have a CoCo 3 with 512K RAM and a CoCo SDC (which lets you keep disk images on an SD card), or have MAME or the VCC emulator set up to emulate a CoCo 3. (On the off chance that you have a Matchbox CoCo, an FPGA-based system that Roger Taylor has created, there's a version for it, too, thanks to the work of Bill Nobel, but for now, the CoCo 3 with CoCo SDCand those two emulators are the supported setups.)
About BASIC09 vs. tokenized BASICs of that era (almost all Microsoft-based, ultimately going back to Altair BASIC): the extreme memory constraints of Altair BASIC (by default the Altair came with 1K of RAM; the first Altair BASIC version needed 4K) forced some design decisions that MS stayed with for a long time, at a catastrophic cost in runtime. There's a series of books, starting with Color BASIC Unravelled, that have a disassembly of the various versions of BASIC that run on the CoCo. (Also, MS has just now opened the source of GW-BASIC, which I think still has the same basic, you should pardon the expression, structure.)
The tokenized MS BASICs of the time make almost no changes to the source code as you type it or load it; lines are put in a singly-linked list with the line numbers stored as two-byte integers. Language keywords are converted to one-byte codes and built-in function names to two-byte codes, but that's it. As lines are interpreted, variable names are looked up by linear search through a symbol table that only keeps the first two characters of variable names (and new entries made when the lookup fails). Every single time a line is interpreted, constants are reconverted from text form to internal format, expressions are reparsed for evaluation. Every GOTO or GOSUB forces a linear search of the statements (though if the number is greater than that of the current line, at least it doesn't have to start from the beginning).
All this means that tokenized BASIC programs of any size fall victim to coding practices that try to reduce the enormous overhead of interpretation at an dire cost in legibility and maintainability. Look at any Color BASIC listing in any Color Computer magazine and you'll see the results:
basic09 (BASIC09's text-oriented IDE, in today's jargon), on the other hand, parses code as you enter it, converting it to I-code. I-code is often likened to Pascal P-code or Java byte-code, but in one way it's quite different, because basic09 has to serve two masters: one is the programmer editing, listing, and debugging code, and the other is the programmer running code. If you have a separate compiler, your byte code can, for example, have simple conditional and unconditional branches like typical computer instruction sets. BASIC09 I-code, OTOH, has constructs that correspond to BASIC09 statements and control structures, so it can, by keeping around symbol table information that is not used at runtime, recreate a very close approximation of the code you enter or load. It's not exact, because
You'll notice I said "procedures". Tokenized BASIC programs of that era are monoliths, "big balls of mud" as Foote and Yoder put it. All variables are global (and with only the first two characters significant in variable names, it's very hard not to reuse the same name by mistake). A BASIC09 program, OTOH, is a set of procedures (which can be machine language or compiled code, as long as it follows BASIC09's calling conventions). BASIC09 procedures have local variables and support recursion; breaking a problem down properly can thus not only make code easier to read, but make it use less memory--important when a process can't get to more than 64K of RAM.
I hope this answers your question and also encourages you to check out BASIC09.
The manual talks about it being a compiler, but I'm not sure I see significant differences between its I-code and tokenized BASICs in general.
Well, so called 'tokenized' BASIC isn't really tokenized, but rather put in a shorthand. Tokenization does require to put everything into token format, which isn't true for average BASICs. The term 'crunched' used by Microsoft is far more appropriated here (*1).
BASIC09 did not only turn recognized keywords into single byte identifiers, keeping the rest (constants/variable names) as is, but puts everything into tokens, which is a virtual machine instruction (*2) as well.
A variable reference is a token of its type (byte/integer/float/string/complex), followed by the address of its value. Similar constants, but with the value following. So unlike 'standard' basics, the interpreter doesn't has to figure out every time what it's about - no matter how lightweight - but simply executes the token.
So far this might look like some sophisticate cruncher, but all operators are as well available in typed version. For example there are
- instructions for byte/integer and float+ instructions for byte/integer, float and strings= instructions for byte/integer, float, strings and bool<> instructions for byte/integer, float, strings and booland so on. Depending on operand types used in an expression BASIC09 will compile the appropriate instruction. This extends to whole BASIC instructions aswell, as there are for example two NEXT instructions for the VM, one that accepts an integer coutner variable and one for float. The later ofc slower.
Next, all expressions are turned into RPN. For example an IF statement will look like this:
(Made arbitrary complex on purpose)
IF MID$(Input$,Pos,1)="\" AND Found=0 THEN
is compiled to
It's quite clear to see that a stack machine is operating along the instruction stream of an expression.
<End of Line> is rather syntactic sugar to insert automatic line breaks (and indentation) when listed, as BASIC09 prefers. Similar all expressions will be 'normalized' and alternate spellings like ><put into their normal form <>.
While I-Code structure is (naturally) close to BASIC, it does as well show differences. For example the TO and STEP parts of a FOR-NEXT are (together with the repeated variable reference) compiled to the NEXT instruction, saving the need to lookup what to examine for end condition and step width. In addition the SEP value will be always compiled as part of the NEXT instruction. As result the FOR statement will only be executed for initialisation. all following iterations will direct jump to the next instruction after FOR (*2).
And so on.
Long story short:
I-CODE is not simply crunched (tokenized) BASIC, but consists of instructions toward a virtual machine, which just happens to be very BASIC compatible.
I know you can PACK it, but that seems more like a symbol stripper.
Yes, it that's what PACK is for. It removes all remaining source reference, like variable names, to save space. In addition it more aggressive optimizes expressions than the interactive interpreter does. After packing a program can only be executed - usually by RUNB.
Keep in mind, OS/9 is a RAM based system and so is BASIC09. BASIC09 (the program) is loaded into memory, thus reducing the RAM available to the user program. RUNB is essentially just the virtual machine (and BASIC runtime). By replacing BASICC09 with the much smaller RUNB more RAM was available by essentially dropping the editor, compiler and debugger parts (*3). IIRC BASIC09 was more than 20 KiB, while RUNB was about half of that. so 10+ KiB more space for a user program is quite a lot.
*1 - Yes, BASIC's come in many colours, including some that normalize way more than MS did.
*2 - This is not done for WHILE loops, as they are rejecting structures, unlike FOR which iterates at least once.
*3 - Well, and even more after packing.
FORX=1TO10) allowed by earlier MS BASICs and BBC BASIC, or join lines, shorten variable names, etc