Was self-modifying code possible using BASIC?

Question

Was self-modifying-code possible using BASIC?

Parameters:

• On commonly affordable home-computers before 1984
• Code that changes its own instructions while it is executing
• Using BASIC only
• Not using peek, poke or assembly language

Possible types of solutions:

Were there any machines on which the following theoretical example program might work?

10 Let Line 20 = Print "word"
20 Print "nothing"

Or just:

10 Let Line 10 = Print "word"

Or the following pseudo-code?

10 Print "10 Print "word"" + [ A special carriage-return that makes the computer accept this as the new 'Line 10' ]

In the example above, if you print the new line 10 to the screen, is there any special carriage-return that makes the computer accept this as the new 'Line 10'?

Note:

• If you did get the computer to modify a line, and you modified the line to be longer than it originally was, could it unknowingly overwrite part of the following line? Outside of program execution mode if you modify a line to be longer than it was, the computer knows how to shift data forward.

I would probably have to ask a separate question to ask:

• How was self-modifying-code used to save space?
• Were there any other interesting uses for self-modifying code that would have been possible on the machines specified in this question?
• Could you somehow set a data file as the new current program?
• How many programs or data files could be held at one time?

____________________________________________

EDIT -
Anyone heard of 'something like' EDITLN or EDITLINE or LINEDIT statement in BASIC for editing lines of a program's own code while executing ?

I seem to remember a statement that was spelled 'something like' EDITLN or EDITLINE or LINEDIT, and wonder if maybe there was one -

• In any version of BASIC
• Used for editing lines of a program's own code while executing

____________________________________________

Answer 1

BBC BASIC, first shipped in 1981, includes the EVAL keyword, which means "ask the interpreter to evaluate this string as an expression". Since strings can be mutated, a program can mutate what will be evaluated at runtime.

The BBC MOS also provides *SPOOL (write screen output to a file) and *EXEC (read text from a file and act as if it had been typed), if you wanted to use secondary store as an intermediary.

A potentially-interesting suggested use of EVAL, at least in the later Z80 port of BBC BASIC, is writing a graph plotter that allows the user to enter a function.

Answer 2

In Commodore BASIC (used in the PET, VIC20, C64, C16, Plus/4, and C128) which is derived from Microsoft BASIC, it is possible to output screen-editor control codes in PRINT statements. Thus a common technique used to create dynamically-generated BASIC lines is to PRINT the generated line followed by control codes to return to the start of the line and then issue a Carriage Return, causing the screen editor to accept the line just as if it had been manually typed. Consider the following 1-line program:

10 PRINT "{CLR-HOME}20 STOP{HOME}{CR}"

When RUN with an appropriate tickle of the keyboard buffer, this will add a new line 20 to the program; execution continues to that new line and stops (this example is trivial, doing no more than simply triggering a ?BREAK IN LINE 20 error).

A classic use-case for this type of self-modifying code is to produce lines containing DATA statements that encode UDG (User Defined Graphics) or Sprite data, or machine code instructions. The resulting code can be appended to a loader stub for easy distribution and re-use.

Answer 3

Given that HP’s Rocky Mountain BASIC could load and run parts of code from disk, and given those parts of code could be plain text and the files could also be saved on the fly, I would say yes it should be possible. I don’t know if anyone tried to do that with RMB though. There were definitely self modifying programs using peek and poke but of course your question rules those out.

As to your second question RMB programs back in the day could only have 31,767/8 (I forget) lines so loading parts from the disk was necessary to save space (after a fashion). The parts on disk were normally already complete pieces though. In fact, TransEra’s HTBASIC (clone of RMB) still works this way (we use it where I work now so I’m still using programming skills I acquired decades ago).

Come to think of it, I also seem to recall a computer that I used (Commodore PET perhaps?) where a program could list part of itself on the screen and then output keystrokes to move the cursor up to a displayed line, make changes and they would be retained- that certainly wouldn’t save any space though.

Answer 4

GW-BASIC allowed you to load in arbitrary code using CHAIN MERGE, where you could take an ASCII-coded (non-binary) BASIC program or snippet and transfer control to a specific line number. As no renumbering occurred, you could overwrite or remove sections of your code prior to the chaining.

This program snippet works in this PC-BASIC emulator:

10 PRINT "Hello"
20 PRINT "world"
30 CHAIN MERGE "MERGE.BAS", 20, ALL
40 GOTO 20

MERGE.BAS (in the current directory):

20 PRINT "DONE"
30 END

When run, it shows:

Hello
world
DONE

Answer 5

The ZX Spectrum has VAL function, which is able to evaluate a string as a numeric expression. Such string can contain any valid BASIC expression. VAL$ does the same but with string expressions.

On the other hand, the SAM Coupé BASIC (influenced by the Sinclair BASIC to some extent) has KEYIN, which allows a BASIC program to interpret a string as a BASIC sentence (not just expression evaluation).

Answer 6

Atari BASIC included the "ENTER" keyword, which was the "opposite of" LIST. Whereas LIST wrote a program to text, ENTER read a program from text. One could use ENTER to read libraries by adding those lines to an existing program.

In contrast to most BASICs, Atari BASIC did not have any limitations between which keywords would operate in interactive mode vs. program mode. Thus:

1 L.

Would cause the program to output itself when run. Likewise, you could use the ENTER in the program to have the program load more code and add it to itself.

Answer 7

First, we must ask "what is self-modifying code?" This can mean a few things.

1. One type is code which modifies the on-disk (or on-tape) version of the code, then re-loads and re-executes it.

   I don't think this is what the question is asking about. I think it wants code that modifies itself in memory, while it is running, without having to reload.

2. Another is code which parses and executes itself (with eval and such), modifying some non-compiled version in memory, then recompiling and re-executing it.

   Again, I don't think this is what the question is asking about. I think it wants code that modifies itself in memory, without having to re-evaluate.

3. Another is to compile the code, and have the compiled assembly modify itself.

   And again, I think this is what the question is specifically attempting to exclude with the "Not using peek or poke or assembly language".

4. Another is to have an interpreted-on-the-fly bytecode language, and have that change the non-compiled code in memory.

   This is how BASIC worked in the Spectrum at least. Each command was a single keystroke (you'd hit P and PRINT would appear). Of the 256 possible character values, 91 were the BASIC commands and operators like POKE or <>, not counting the single-character operators like + and so on. You can see the list on Wikipedia.

   I think the question makes an incorrect assumption about peek and poke: that they are used to hack in funky assembler codes or something.

   So, the number 1 took four bytes to store on the Spectrum (if I remember right, it stored two copies of all 16 bit ints for some reason), but the commands SGN PI took only 2, one byte each: 0xBC 0xA7.

   In BASIC, the commands to change the contents of memory, the equivalent of C's pointers, are PEEK and POKE.

   It makes no sense to ask for self-modifying, non-reloading in-memory code, then saying that you can't use the specific commands that would access the in-memory commands you would want to change.

5. A final option, suggested by @Eric Towers, is some system that edits at a metasyntactic level.

   Again, I don't think this is what the question is asking. While BASIC (like any language, really) can be compiled to bytecode or assembly, it was typically implemented as a dynamically interpreted language from a byte stream. Outside of the byte stream, there's no pre-parsing, splitting the language up into metasyntactic concepts akin to a "syntax tree", or anything like that.

   As I understand it, Sinclair BASIC in particular was an interpreted language: the program counter went along line by line, passing each in turn to the interpreter, in a single pass with no backtracking other than via the GO TO construct.

   Because of this, the closest to a metasyntactic command would be line-number based commands, such as:

   • the Timex Sinclair's addition to the Sinclair BASIC of the DELETE <first line#> <last line#> command permitted the removal of ranges of lines from a BASIC program.
   • the RENUM <first line$> <step size> command renumbered code line numbers, affecting all GOTO, GOSUB, RESTORE, RUN, LINE, ON ERROR GOTO and other similar commands. The reason this could be considered self-modifying is that it would leave alone any expressions such as GOTO VAL "50", DELETE 10, 100, and indeed further calls to RENUM.

   So it's possible, though I've not tried it, that this:

   10 GOTO VAL "40"
   20 PRINT "nothing"
   30 DELETE 10, 70
   40 PRINT "word"
   50 RENUM 10, 30
   60 DELETE 130, 160
   70 GOTO 10
   

   ...will, on hitting line 50, change itself to:

   10 GOTO VAL "40"
   40 PRINT "nothing"
   70 DELETE 10, 70
   100 PRINT "word"
   130 RENUM 10, 30
   160 DELETE 130, 160
   190 GOTO 10
   

   ... then on hitting the new line 160, convert to:

   10 GOTO VAL "40"
   40 PRINT "nothing"
   70 DELETE 10, 70
   100 PRINT "word"
   190 GOTO 10
   

   ... then on hitting the new line 70, convert to:

   100 PRINT "word"
   160 GOTO 10
   

   ... which is still valid since a GOTO goes to the next largest number. The resulting output would be:

   word
   nothing
   word
   word
   word
   ....
   

   I have never used a Timex, so can't say for sure that's how they worked (did these update the program counter? Halt the program? Restart it? I can't find it documented). But either way, this is the closest I can find.

Answer 8

Applesoft BASIC, as I recall, treated ASCII character 4 (control-D) as an escape code to DOS or ProDOS. You could therefore give the interpreter commands such as PRINT CHR$(4)"WRITE filename", and to run a different program.

Answer 9

Program code as data file

Microsoft Basic, e.g. on 8 bit Commodore machines, supported reading and writing program files on floppy disk like sequential data files.

OPEN 2,8,2,"EXAMPLE,P,W"

would open a file named EXAMPLE with type PRG for writing. Then one could create a program with a couple of PRINT# instructions, and CLOSE it, it'd show up as an executable file on disk. It should be written as it'd appear on the memory, i.e. line numbers as 16 bit integers, keywords should be tokenized etc.

Chain-loading or chaining

The LOAD instruction, when issued in a running program

999 LOAD "EXAMPLE",8

loads the named program from disk, replacing the running one, and starts executing it at the first instruction. As long as the loaded program is smaller than the first one, variables would be preserved, so they can communicate that way with each other.

Read-modify-write

A program could as well open the file containing its own code, read it one line at a time, modify some stuff on the fly, write it back to another file, then chain-load it.

Answer 10

Beta Basic for the ZX Spectrum had ALTER, which was specifically intended to "alter references in the program" as demonstrated by this code:

1 LET free=MEM()  
2 FOR n=1 TO 100 
3 ALTER (n) TO "VAL"+CHR$ 34+STR$ n+CHR$ 34  
4 NEXT n 5 PRINT "Saved ";MEM()+28-free;" bytes"

This replaces all numbers in the program from 1 to 100 with a variable. The manual also warns about careless usage of ALTER:

NOTE: Be cautious when using ALTER. You can make irreversible changes to a program if you are careless. For example, if you change all uses of the numeric variable "apple" to "a" and then realise that you have already used the variable name "a", you cannot simply alter all uses of "a" to "apple" to rectify matters! You should have checked for previous use of "a" using the REF or LIST REF command.

Answer 11

When I came across an article about self modified Basic code on Mega65 project's blog. I wanted to try it myself and wrote the following code. The program itself doesn't make much sense but it works.

The trick is to place erroneous syntax in Basic memory. Then find the address of the erroneous tokens to replace it with whatever you want in runtime.

The code is written for CBM Basic 2.0 (C64) in petcat format.

10 gosub 9000 : rem find gosub address
20 print "{clear}{white}"
30 print "self modifyng basic"
40 ln=1000 : gosub 9100 : rem update address
50 gosub,12345
60 goto 50

999 rem ******************* subroutines
1000 print "this subroutine runs only once"
1010 ln=2000 : gosub 9100
1020 return

2000 print "this subroutine runs only once, too"
2010 ln=3000 : gosub 9100

3000 poke ja,44 : rem place comma again 
3010 end

8999 rem
9000 rem *********** find gosub address
9010 for ja=2048 to 40959
9020 if peek(ja-1)<>141 or peek(ja)<>44 then next
9030 return
9100 rem ****** change gosub address
9110 ln$=str$(ln)
9120 for i=0 to 5: poke ja+i,32 : next
9130 for i=1 to len(ln$): poke(ja+i),asc(right$(left$(ln$,i),1)) : next
9140 return

Blog post I've mentioned is in http://c65gs.blogspot.com/2018/04/gosub-variable-in-c64-basic.html

Answer 12

The Sinclair QL updated with SMSQ/E and some BASIC extensions (all courtesy of about 1990) does that in a very straightforward way by using two multi-tasking BASIC jobs, one typing in code to the other:

100 EXEP "SBASIC" : REMark runs a daughter BASIC job
110 TYPE_IN "100 PRINT 'Hello, World'" & CHR$(10) : REMark TYPE_IN sends keystrokes from string to the current console
120 TYPE_IN "110 GO TO 10" & CHR$(10)
130 TYPE_IN "RUN" & CHR$(10)

This starts a second instance of BASIC, multitasking with the current one and "types in" a program, then runs it. Pretty much like in your hypothetical example.

Answer 13

Example of using self-modifying code to save space and time: (or, actually, to overcome the limitations of the built-in interpreter)

In the C64 BASIC there is no way to evaluate string as formula (unlike, say, ZX Spectrum' VAL()). So in a function plotter program you have to either implement your own parsing and evaluation (which can be quite complex and inefficient in basic) or make your program incorporate the user-provided formula into itself, for example as a line similar to 1000 DEF FN F(X)=3*SIN(X/10)

Answer 14

The ZX Spectrum had the MERGE statement, which would read a saved program from a file and create or overwrite corresponding lines in the current program. (Any saved variables were also merged in the same manner.) However the program had to be in the ZX Spectrum's internal SAVE format, so you couldn't readily use it for arbitrary code. By default the program would load from tape, which made it inconvenient to use programmatically, but the Interface 1 device may at least have made it possible to use the MERGE statement as an overlay loader (I never had one though).

This doesn't directly count as it involves assembly language, but I wrote my own extensions to ZX Basic, one of which allowed the program to merge from a text file input stream instead. (As I recall you had to remember to include a GO TO statement in the text to resume code execution.)

Answer 15

Yes. In tokenized Basic interpreters (e.g. many early MicroSoft Basic derivatives) that include peek() function and poke statement, a running programs could examine its own tokens by peeking at memory, and modify the tokens on-the-fly by poking new values of tokens to be later executed.

The token formats for various statement and operations were reverse-engineered, with tables published and/or passed around various user groups.

This technique allowed, for instance, changing the expression A+B to A-B (etc.) for a larger but tight graphics loop without the overhead of an "IF" statement inside each loop iteration or repeating lots of code.

Answer 16

Chipmunk Basic supports the merge command.

Running this program

10 merge "foo.bas"

with a file including this text (initially not in the Basic program):

20 print "hello 20"

will print

hello 20

Having a program write a .bas file before using the merge command on it allows even more modification.

Answer 17

One of the advanced techniques used on the TRS-80 was to build text strings containing character codes that represented Z80 machine language. These text strings were manipulated and combined and then executed from BASIC. Here is a typical sample from the popular game Android Nim:

9000 HA$=CHR$(160)+CHR$(181)+CHR$(186)+CHR$(144) 
9001 HB$=CHR$(128)+CHR$(143)+CHR$(182)+CHR$(185)+CHR$(143)+CHR$(128) 
9002 H1$=CHR$(128)+CHR$(143)+CHR$(183)+CHR$(187)+CHR$(143) 
9005 HC$=CHR$(160)+CHR$(186)+CHR$(176)+CHR$(149) 
9006 HD$=CHR$(128)+CHR$(143)+CHR$(189)+CHR$(179)+CHR$(142)+CHR$(128) 
9007 H2$=CHR$(128)+CHR$(143)+CHR$(191)+CHR$(179)+CHR$(143) 
9010 HE$=CHR$(160)+CHR$(176)+CHR$(181)+CHR$(149)

This would definitely qualify as self-modifying code. After running the program, if you did a LIST instruction, you would see strange characters where the code was modified.

Answer 18

In the APPLE II, the DOS 3.3 had a program called Phone List which stored the data of the phone numbers by modifying itself. It had empty DATA instructions from the lines 200 to 400 and this instructions would get filled when one entered new records. Upon program termination a SAVE would be executed to save the contents of the program again. Next time the program was loaded, the data would be read directly with READ instructions.

Answer 19

If your willing to write your own interrupter then yes, but with specific limitation due to ram.

So lets say we have an AI.CMD file. Open,reading,writing, add lines, deleting line from any open file, were all part of the standard basic language.

Since we are talking about re-writing for an AI/self modifying program we are doing away with lengthy text descriptions.

General logic not fully executable code.

10 open #1,"AI.CMD"
20 input #1,VLINE,COMMAND$,PARAMETERS$
30 if AI(VLINE,COMMAND$,PARAMTERS$)<>0 then goto 20 (the AI decide not to run this line of code)
30 if COMMAND$="1" then print PARAMETERS$
40 if command$="2" then duplicate a different command

So every BASIC command would be assigned a number, and would get its own IF statement. So here the print command is represented as command #1. I am not saying this is going to be easy by any means.

Using the standard commands you can now modify the AI.CMD file.

Now deciding what to modify when is an incredible complex task.

As far as I know nobody actually did this.

How many programs or data files could be held at one time?

Generally, due to limited RAM, no files were held in memory at once, but instead a small pointer to said file was stored in memory. The limits depended on the version of BASIC, but you could have 8 files or more open.

---

Generally, your biggest problem is running out of memory to store the program. I started writing a TI Extended BASIC to assembly language convertor, but after getting even part way through I kept getting out of memory errors. The TI 99/4(A) only had 32K memory with a massive expansion box and memory card. So that ended that project.

The key here is the size and complexity of the self modifying routine would be very limited due to RAM size. Computers 1984 and before, didn't really have enough functionality to make attempting this practical.

Answer 20

I seem to remember a statement that was spelled 'something like' EDITLN or EDITLINE or LINEDIT,

TL;DR: There was EDIT in many (MS-)BASICs, but it was a direct command, not applicable during runtime.

---

Long Story:

I guess that memory is about EDIT <line>. A command present since Altair BASIC, but optional, depending on the philosophy and space. It was left out in PET BASIC due it's screen editor (*1), and of Applesoft to make room for additional commands (*2). It is for example present in Tandy's Level II Basic (based on BASIC 80) and ColourBASIC (BASIC 69). It was supported all the way to BASIC 5.28 and thus GW-BASIC as well. Some systems (like IBM's BASICA) kept it, despite the existence of a screen editing feature.

EDIT is part of the program editing commands like NEW, DEL or AUTO.

It was meant to support editing on dumb, teleprinter level terminals. It invoked a line editor for a single (existing) BASIC line given as parameter. The cursor was moved by Space and Backspace. Several commands were available to overwrite, insert and delete characters - including repetition.

---

*1 - Not really a screen editor in today's understanding, but rather working on logical lines displayed and the ability to change one and hand it over to BASIC again like a newly entered one.

*2 - A 'shadow cursor' handling allowed to copy (parts of) lines from screen into the promt line.

Answer 21

On the Sinclair ZX Spectrum, if you know where the BASIC program resides in memory, and you know the format of the tokenized program, you can modify the code by POKEing into this memory.

And you can always find the location of the program by looking at the system variable called PROG at address 5C53h with the command PRINT PEEK 23635. This address was typically 5CCBh on original 48K speccies but could and did move around depending on what peripherals were attached.

A more common use of this technique that I wouldn't consider true self-modification, is to have a very small Program, typically one line with a RANDOMIZE USR <nnnn> statement followed after a colon by a REM statement followed by a chunk of text. Then you POKE some machine code bytes over the exact place where the chunk of text after the REM is and use that address where I put the placeholder.

Another thing you can do in Speccy Basic is hide the true value of numeric literals. In the tokenized version of Basic each integer literal is stored twice, first as an ASCII string of digits, then as a six-byte field, where two bytes hold the number as a 16-bit integer. You can use POKE to change that 16-bit field to the value you really want where naive users trying to LIST your program will still see the ASCII red-herring values.

The first is more of a memory saving hack and the second is more of a naive software protection method, than actual self-modifying code. But using the exact same technicque you could have a program that POKEd arbitrary new code right over existing code, and the interpreter will just interpret the new token stream when it gets to it.

I'm not sure if anyone actually did this. When you got to this level in Speccy Basic you were probably moving on to program in machine code anyway. But I remember discovering it as a kid in the early '80s so I'm sure people found ways to abuse it.

Answer 22

MERGE and the Cassette Buffer File/Tape Buffer
Many or most versions of BASIC on these machines would have had a "Cassette Buffer File/Tape Buffer", and many would also have had a command like MERGE .
. . .So maybe the commands OPEN "CAS: PROGRAM" FOR OUTPUT AS #1 and PRINT #1,"10 PRINT 1"or one of the Save to Cassette commands could write a program line ( or short program ) to that "Cassette Buffer File/Tape Buffer", and then using the MERGE command to put( or replace ) a line in the main current program with what is in the tape buffer.
. . . NOTE - Obviously I mean using the tape buffer without even needing to plug in a tape-machine .
( Interestingly, some programs for the c-64 were stored on and ran from the "Cassette Buffer File/Tape Buffer" . )

AUTO
GW-BASIC and others had a command like AUTO which generates new line numbers ( so you don't have to type them ), but I don't know if it could be used from a program.
. .If AUTO would function from a program, then maybe lines could be replaced or added ( using AUTO and the PRINT command to enter text and ENTER )

NOTE - I posted another question at the same time as this one asking how many programs could be on these machines at the same time / concurrently, and one of the answers would have been identical to this answer I have posted ( although at the time I either just wasn't fully believing that tape-buffers etc existed, or wondered if it worked another way ). Unfortunately the other question I posted has been totally deleted of this site, not by me, I would be happy to delete my answer, maybe an issue for admin