How were binary subroutines embedded in Integer BASIC programs?

Question

Programs written in Apple's Integer BASIC sometimes had bits of machine code embedded in them. This was more convenient than loading the code separately, especially when the program was loaded from tape.

When the program is first loaded, it looks garbled. For example, the classic APPLE-VISION demo starts like this:

    0 LOMEM:1800
    1 HIMEM:8192: POKE 202,165: POKE 203,27:LOMEM:6144:CLR : GOTO 0
 8264, RUN LOMEM:*2664439008, ^ , RUN , ? ABS *2664439008, ^ , RUN ABS *266438288; RUN ^ , RUN 6164,=58882/42004#2112 24576*

The LIST command would sometimes go into an infinite loop. But after the program is first run, the listing looks more normal:

    0 REM  *** APPLE-VISION ***  WRITTEN BY R.J. BISHOP
   20 DIM A$(51)
   30 A$="       BOB BISHOP PRESENTS . . . APPLE-VISION!       "
   40 CALL -936

How does this work, in terms of both the underlying mechanism and the mechanics of developing a program that hides code this way? e.g. how does one combine the BASIC and machine code portions before saving to disk or tape?

Update 1: for the curious, a commented disassembly of APPLE-VISION can be found here.

Update 2: another program by Bob Bishop, called Bomber, was written entirely in assembly language, but used an Integer BASIC wrapper. As far as I can tell, this was done solely to make it easier to play the game from cassette tape: you could read it in with a LOAD command from BASIC, instead of having to drop into the monitor and specify an address range. Full disassembly of Bomber is here.

Answer 1

In this Integer BASIC disassembly, 202/203 ($CA/$CB) is PP - "ptr: start of program". It looks like they're changing the "start of program" location, then re-running it by doing a GOTO 0.

My guess is that there's a small BASIC program (the one you see at the start), then a chunk of assembly language that looks like junk when listed, then another small BASIC program (the one you see at the end).

What I'm not sure is why they don't just jump directly into the assembly routines… I might have to investigate further :-)

Assembly language was often included in BASIC programs though: although the S. H. Lam Technique is for Applesoft BASIC, a slight variation works fine in Integer BASIC. It fills the keyboard buffer with keystrokes, then calls the monitor parse-and-process routine to execute the keystrokes as a monitor command.

10 DIM H$(23):H$="300:AD 30 C0 60 N D9C6G"
20 FOR X=1 TO LEN(H$): POKE 511+X, ASC(H$(X,X)): NEXT X
30 POKE 72,0: CALL -144
40 PRINT "HELLO"
50 END

Other programs found creative ways of embedding actual assembly code somehow into a BASIC program. Ivan Drucker's KansasFest talk on Slammer discusses a clever, modern variant of this technique.

Answer 2

Generally, this involves some very careful work, where you overwrite part of the tokenized BASIC program with the binary data, or add the binary data to the tokenized BASIC code while carefully adjusting the counters showing where things are located in the code. It's a tricky process and I never thought it was worth the trouble, given how easy it is to mess up.

It does make it harder to edit the BASIC code, since it becomes problematic for the BASIC interpreter to revise the code as you make changes when it can't understand the "tokenized code" which is actually binary data. That makes it a crude copy protection scheme both in terms of obfuscating the code and in terms of making it harder to change the code without damaging it.

I don't have the details on how to pull this off handy; it can be figured out by looking at the internal format of the tokenized BASIC code, though. Once you know how Integer BASIC (and Applesoft, for that matter) store their code, you can start to manipulate it to do pretty clever things. It takes patience and practice though.

Answer 3

The technique that zellyn describes above for having the Monitor process a command from the input buffer was, as he said, devised for Applesoft. In fact, one of its crucial elements is specific to that language: the "D9C6G" references the Applesoft ROM code. As it happens, the file that was often distributed to provide Integer BASIC for the Language Card had, of course, Integer BASIC itself, plus the mini-assembler and Sweet-16 interpreter, at $E000 to $F7FF, with "Programmer's Aid #1" added at $D000 to $D7FF, and $D800 to $DFFF left with simply a copy of the Applesoft ROM code of the same address space. Thus, the routine at $D9C6 was present, and the above-listed technique would work. However, this would not be reliable -- the $D800-$DFFF space might not have a copy of that portion of Applesoft; and in particular, if the above is tried on an original-model Apple ][, with Integer BASIC itself in ROM and no trace of Applesoft, it definitely won't work. There is an address intrinsic to Integer BASIC that does the same job: E817. Use that in place of D9C6 for something that will reliably work.

A second, minor point is that the "N" just before the D9C6/E817 is used to safely terminate the memory-assignment command. That's usually fine, as it just sets text to "Normal" display mode. However, if one happened to have been using Inverse mode, this would cancel it. A completely generalized method would be to replace the "N" with a "<" directly prefixing the E817G: "<E817G".

More significantly, however, a faster technique for storing the string in the input buffer, compared to a BASIC poke-loop, would be to "trick" Apple's DOS 3.3 into storing the string in the input buffer for you, by PRINTing it as a simulated DOS command. (The bold-italic D is an embedded "ctrl-D"):
10 PRINT "D 300:AD 30 C0 60 <E817G";:POKE 513,141:PRINT: POKE 49,0:POKE 52,2:CALL -123
In the above line, the PRINT begins with a "ctrl-D" to indicate a DOS command. This causes DOS to store all that is printed in the input buffer, instead of actually printing it. The POKE 513,141 then replaces the second character, which was a SPACE, with a <Return> in the buffer. Then PRINT actually generates a <Return>, to let DOS process the "command". Thanks to the above POKE, DOS sees only a "null" command, and does nothing more. Then the two POKEs and CALL set up the monitor and have it process the printed string as a command.

Answer 4

I can't speak directly to Apple Integer BASIC, but I know that TRS-80 BASIC programs could encode machine language in DATA statements (as in Leo Christopherson's Android NIM) or even in literal strings. A string containing machine code would include lots of non-printing characters that would do things like moving the cursor backward or up, or clearing the screen, so the listing would look garbled.

If the program shows up normally after it's run, that probably indicates that it's self-modifying; again, the Android NIM page above mentions this.

As for how to do it: You could write code in assembly language, run it through an assembler, and then insert the machine code into strings or data statements, but these machines were so small and simple that an ambitious programmer could compose machine code directly.

Answer 5

Bob Bishop helped us with a Machine Control program using the Apple ][ as a Machine Controller with GREAT Graphics. Late '70 and early 80's He showed me the trick where you insert the Machine Code in a REM Statement of the First line of basic, then modify the Line code to say it at the end. I forget the actual Tokens used.. I have them somewhere in a Stack of notes. The list will not show the Garbage as the list stopped looking at the line, and the Next Line Memory location Pointer showed the Start of the next printed line. If the REM statement you have modified is the First line of the Program, and the First Line is just a REM Statement, then it's always in the same place in memory to execute a Call to.