24

Was self-modifying-code possible using BASIC?

Parameters:

  • On commonly affordable home-computers before 1984
  • Code that changes its own instructions while it is executing
  • Using BASIC only
  • Not using peek, poke or assembly language

Possible types of solutions:

Were there any machines on which the following theoretical example program might work?

10 Let Line 20 = Print "word"
20 Print "nothing"

Or just:

10 Let Line 10 = Print "word"

Or the following pseudo-code?

10 Print "10 Print "word"" + [ A special carriage-return that makes the computer accept this as the new 'Line 10' ]

In the example above, if you print the new line 10 to the screen, is there any special carriage-return that makes the computer accept this as the new 'Line 10'?

Note:

  • If you did get the computer to modify a line, and you modified the line to be longer than it originally was, could it unknowingly overwrite part of the following line? Outside of program execution mode if you modify a line to be longer than it was, the computer knows how to shift data forward.

I would probably have to ask a separate question to ask:

  • How was self-modifying-code used to save space?
  • Were there any other interesting uses for self-modifying code that would have been possible on the machines specified in this question?
  • Could you somehow set a data file as the new current program?
  • How many programs or data files could be held at one time?

____________________________________________

EDIT -
Anyone heard of 'something like' EDITLN or EDITLINE or LINEDIT statement in BASIC for editing lines of a program's own code while executing ?

I seem to remember a statement that was spelled 'something like' EDITLN or EDITLINE or LINEDIT, and wonder if maybe there was one -

  • In any version of BASIC
  • Used for editing lines of a program's own code while executing

____________________________________________

  • 12
    Most BASICs I worked with had a "NEW" command, which would delete the current program. Can't get more drastically self-modifying than that... – Spacedman Jun 8 at 12:28
  • 2
    In DTSS (=the original home of BASIC), "NEW" etc., as well as the convention that numbered lines got added to the working file, were part of the command line interpreter, not BASIC. Subsequent BASIC subsystems tended to embed DTSS-alike commands in the same program as the BASIC interpreter/compiler, but IMO they're still "not BASIC". And anyway, could a BASIC program issue a "NEW" command? If not, it's not self-modifying :-) – another-dave Jun 8 at 15:14
  • Poke commands affecting the stored listing are implementation dependent. Self modification in a language is generally by design. – Mark Williams Jun 8 at 17:17
  • 6
    @MarkWilliams Any non-trivial BASIC program is implementation dependent. – Ross Ridge Jun 8 at 17:24
  • I've seen it done on the Atari 800; I don't recall how it worked, but I know it's possible. – Thomas Jun 8 at 19:02

17 Answers 17

40

BBC BASIC, first shipped in 1981, includes the EVAL keyword, which means "ask the interpreter to evaluate this string as an expression". Since strings can be mutated, a program can mutate what will be evaluated at runtime.

The BBC MOS also provides *SPOOL (write screen output to a file) and *EXEC (read text from a file and act as if it had been typed), if you wanted to use secondary store as an intermediary.

A potentially-interesting suggested use of EVAL, at least in the later Z80 port of BBC BASIC, is writing a graph plotter that allows the user to enter a function.

  • I regard "self modifying code" as quite different from "compile and execute new code". Could EVAL be used to change code that was part of the executing program (as opposed to changing the program text on disk, so that the change had to wait for the next reload of the program)? Even CHAINing to the new code doesn't seem like it's really self-modifying. – another-dave Jun 8 at 15:21
  • @another-dave obviously it's BASIC, so it's never compiled, but pedantry aside: EVAL evaluates a string from directly in memory and does whatever the interpreter would do if a user had typed it interactively. Very much like Javascript's eval(). The disk isn't involved; the *SPOOL/*EXEC suggestion (which, technically, isn't part of the BASIC but rather the OS underneath) was a separate suggestion, flowing from Michael Tracy's observations of HP BASIC. – Tommy Jun 8 at 17:43
  • 3
    @Tommy The original DTSS BASIC was compiled, and many PC implementations compiled to byte code. – Barmar Jun 8 at 19:40
  • @Barmar ugh, clearly my statement is incorrect. My specific failure was switching train of thought mid-sentence, not that it helps to say so: "it's never compiled" is patently false. – Tommy Jun 8 at 19:54
  • 2
    Can one EVAL a string with a line number to change the currently running program? – Max Jun 10 at 19:39
20

Given that HP’s Rocky Mountain BASIC could load and run parts of code from disk, and given those parts of code could be plain text and the files could also be saved on the fly, I would say yes it should be possible. I don’t know if anyone tried to do that with RMB though. There were definitely self modifying programs using peek and poke but of course your question rules those out.

As to your second question RMB programs back in the day could only have 31,767/8 (I forget) lines so loading parts from the disk was necessary to save space (after a fashion). The parts on disk were normally already complete pieces though. In fact, TransEra’s HTBASIC (clone of RMB) still works this way (we use it where I work now so I’m still using programming skills I acquired decades ago).

Come to think of it, I also seem to recall a computer that I used (Commodore PET perhaps?) where a program could list part of itself on the screen and then output keystrokes to move the cursor up to a displayed line, make changes and they would be retained- that certainly wouldn’t save any space though.

  • 1
    32767 lines, probably :-) – another-dave Jun 8 at 15:23
  • 3
    i can recall the trick of self-modifying program by printing BASIC lines and simulating RETURN keystrokes in Commodore 64. note that while we had to POKE to put RETURN in the keyboard buffer, the program structure itself was not manipulated by POKE. – szulat Jun 8 at 17:11
18

In Commodore BASIC (used in the PET, VIC20, C64, C16, Plus/4, and C128) which is derived from Microsoft BASIC, it is possible to output screen-editor control codes in PRINT statements. Thus a common technique used to create dynamically-generated BASIC lines is to PRINT the generated line followed by control codes to return to the start of the line and then issue a Carriage Return, causing the screen editor to accept the line just as if it had been manually typed. Consider the following 1-line program:

10 PRINT "{CLR-HOME}20 STOP{HOME}{CR}"

When RUN with an appropriate tickle of the keyboard buffer, this will add a new line 20 to the program; execution continues to that new line and stops (this example is trivial, doing no more than simply triggering a ?BREAK IN LINE 20 error).

A classic use-case for this type of self-modifying code is to produce lines containing DATA statements that encode UDG (User Defined Graphics) or Sprite data, or machine code instructions. The resulting code can be appended to a loader stub for easy distribution and re-use.

  • 4
    You missed an important step: poking characters into the keyboard buffer (poke 198 to set the count, and 631-640 to set the actual characters). Code would typically print the code two lines from the top of the screen, followed by a "RUN [someLine]", and then poke enough returns into the buffer for all the lines and the RUN, then home the cursor and exit. – supercat Jun 8 at 17:30
  • It would also have been possible, and not even very difficult, to use poke to modify a program that was running, provided that before creating any variables one would need to keep, one adjusted the end-of-program address as needed and then did a CLR. – supercat Jun 8 at 17:32
  • 1
    Using POKE is explicitly excluded in the question. – berendi Jun 8 at 19:35
  • @supercat I wasn't really going for a complete 'how-to' tutorial, merely trying to outline the basic (hah!) principle behind the technique. – Eight-Bit Guru Jun 9 at 19:35
  • 1
    @berendi: Outputting to the screen wouldn't achieve automatic self-modification unless one POKEs keystrokes into the buffer as mentioned. – supercat Jun 9 at 22:23
14

GW-BASIC allowed you to load in arbitrary code using CHAIN MERGE, where you could take an ASCII-coded (non-binary) BASIC program or snippet and transfer control to a specific line number. As no renumbering occurred, you could overwrite or remove sections of your code prior to the chaining.

This program snippet works in this PC-BASIC emulator:

10 PRINT "Hello"
20 PRINT "world"
30 CHAIN MERGE "MERGE.BAS", 20, ALL
40 GOTO 20

MERGE.BAS (in the current directory):

20 PRINT "DONE"
30 END

When run, it shows:

Hello
world
DONE
  • Anything that has MERGE allowed in programs could do that. And I think that is the most practical way of using self-modifying code in BASIC. – tofro Jun 9 at 8:23
  • Something very unusual I just found for MSX. quote -"The new program created by this merging stays in the MSX memory and can be handled as another BASIC program (especially, you can run it, save it, modify it)" - msx.org/wiki/MERGE – questiontype Jun 9 at 15:21
  • 1
    My reading of that text from the wiki is that the new program (ie, original program plus merged lines) has now replaced the original in memory, and can be treated like any other BASIC program. – john_e Jun 9 at 20:06
9

The ZX Spectrum has VAL function, which is able to evaluate a string as a numeric expression. Such string can contain any valid BASIC expression. VAL$ does the same but with string expressions.

On the other hand, the SAM Coupé BASIC (influenced by the Sinclair BASIC to some extent) has KEYIN, which allows a BASIC program to interpret a string as a BASIC sentence (not just expression evaluation).

  • 1
    Also, if you save a basic program after running it, all the variable values from its previous execution are saved along with the code. It's not related to this question, but ZX Basic has another quirk that If statements on the same line separated by colons function similar to nested blocks in more advanced languages. – Paul Humphreys Jun 11 at 1:00
  • same as BBC BASIC's EVAL above. Not surprising, as Clive was going after the BBC educational contract too. – scruss Aug 6 at 12:28
7

Atari BASIC included the "ENTER" keyword, which was the "opposite of" LIST. Whereas LIST wrote a program to text, ENTER read a program from text. One could use ENTER to read libraries by adding those lines to an existing program.

In contrast to most BASICs, Atari BASIC did not have any limitations between which keywords would operate in interactive mode vs. program mode. Thus:

1 L.

Would cause the program to output itself when run. Likewise, you could use the ENTER in the program to have the program load more code and add it to itself.

  • If POKE is allowed, then in addition Atari BASIC allowed you to print a line (or multiple) of text on the screen, followed by a bare line with CONT, position the cursor at the line, put the IOCTL system into Return Key mode (POKE 842,13), and then execute a STOP statement. The line under the cursor would be entered as though from a keyboard and program execution resumed. – Clinton Pierce Aug 8 at 18:19
6

Applesoft BASIC, as I recall, treated ASCII character 4 (control-D) as an escape code to DOS or ProDOS. You could therefore give the interpreter commands such as PRINT CHR$(4)"WRITE filename", and to run a different program.

5

First, we must ask "what is self-modifying code?" This can mean a few things.

  1. One type is code which modifies the on-disk (or on-tape) version of the code, then re-loads and re-executes it.

    I don't think this is what the question is asking about. I think it wants code that modifies itself in memory, while it is running, without having to reload.

  2. Another is code which parses and executes itself (with eval and such), modifying some non-compiled version in memory, then recompiling and re-executing it.

    Again, I don't think this is what the question is asking about. I think it wants code that modifies itself in memory, without having to re-evaluate.

  3. Another is to compile the code, and have the compiled assembly modify itself.

    And again, I think this is what the question is specifically attempting to exclude with the "Not using peek or poke or assembly language".

  4. Another is to have an interpreted-on-the-fly bytecode language, and have that change the non-compiled code in memory.

    This is how BASIC worked in the Spectrum at least. Each command was a single keystroke (you'd hit P and PRINT would appear). Of the 256 possible character values, 91 were the BASIC commands and operators like POKE or <>, not counting the single-character operators like + and so on. You can see the list on Wikipedia.

    I think the question makes an incorrect assumption about peek and poke: that they are used to hack in funky assembler codes or something.

    So, the number 1 took four bytes to store on the Spectrum (if I remember right, it stored two copies of all 16 bit ints for some reason), but the commands SGN PI took only 2, one byte each: 0xBC 0xA7.

    In BASIC, the commands to change the contents of memory, the equivalent of C's pointers, are PEEK and POKE.

    It makes no sense to ask for self-modifying, non-reloading in-memory code, then saying that you can't use the specific commands that would access the in-memory commands you would want to change.

  5. A final option, suggested by @Eric Towers, is some system that edits at a metasyntactic level.

    Again, I don't think this is what the question is asking. While BASIC (like any language, really) can be compiled to bytecode or assembly, it was typically implemented as a dynamically interpreted language from a byte stream. Outside of the byte stream, there's no pre-parsing, splitting the language up into metasyntactic concepts akin to a "syntax tree", or anything like that.

    As I understand it, Sinclair BASIC in particular was an interpreted language: the program counter went along line by line, passing each in turn to the interpreter, in a single pass with no backtracking other than via the GO TO construct.

    Because of this, the closest to a metasyntactic command would be line-number based commands, such as:

    • the Timex Sinclair's addition to the Sinclair BASIC of the DELETE <first line#> <last line#> command permitted the removal of ranges of lines from a BASIC program.
    • the RENUM <first line$> <step size> command renumbered code line numbers, affecting all GOTO, GOSUB, RESTORE, RUN, LINE, ON ERROR GOTO and other similar commands. The reason this could be considered self-modifying is that it would leave alone any expressions such as GOTO VAL "50", DELETE 10, 100, and indeed further calls to RENUM.

    So it's possible, though I've not tried it, that this:

    10 GOTO VAL "40"
    20 PRINT "nothing"
    30 DELETE 10, 70
    40 PRINT "word"
    50 RENUM 10, 30
    60 DELETE 130, 160
    70 GOTO 10
    

    ...will, on hitting line 50, change itself to:

    10 GOTO VAL "40"
    40 PRINT "nothing"
    70 DELETE 10, 70
    100 PRINT "word"
    130 RENUM 10, 30
    160 DELETE 130, 160
    190 GOTO 10
    

    ... then on hitting the new line 160, convert to:

    10 GOTO VAL "40"
    40 PRINT "nothing"
    70 DELETE 10, 70
    100 PRINT "word"
    190 GOTO 10
    

    ... then on hitting the new line 70, convert to:

    100 PRINT "word"
    160 GOTO 10
    

    ... which is still valid since a GOTO goes to the next largest number. The resulting output would be:

    word
    nothing
    word
    word
    word
    ....
    

    I have never used a Timex, so can't say for sure that's how they worked (did these update the program counter? Halt the program? Restart it? I can't find it documented). But either way, this is the closest I can find.

  • 1
    Regarding your conclusion: No. Lisp is an example of self-modifying, non-reloading, in-memory code. The advantage list has is that its one datatype is also its representation for code. That is, the language has high-level constructs to modify its ASTs (abstract syntax trees), however represented. As you have observed, few (perhaps no?, this being the Question) BASIC has high-level constructs to modify its ASTs. – Eric Towers Jun 8 at 19:04
  • @EricTowers Not convinced that your interpretation is the one intended, but it's certainly a valid option 5: I've edited with a suggestion from TIMEX T/S 2000 BASIC. But I've no idea if it'd work in practice. – Dewi Morgan Jun 8 at 23:06
  • 1
    While some versions of BASIC may include statements such as RENUM which alter the program in memory, the act of modifying a program via means other than POKE will cause most interpreters to forget what part of the code was being executed. – supercat Jun 10 at 19:47
  • @supercat without having one in front of me, I can't tell. I can't find anything for the Timex' BASIC which says whether it does or does not update the program counter with the editor commands. If it halts or restarts the program I agree that it likely wouldn't count as self-modifying. – Dewi Morgan Jun 10 at 23:17
  • 1
    @DewiMorgan: I'd count it as self-modifying if it restarts, but the range of what can be done would seem rather limited. It would be interesting if a version of BASIC kept a start-of-variables pointer separate from end-of-program, could leave variables undisturbed on an edit if there was adequate space for the edit, and included an EDITLINE n,"string" command to modify a line of code directly and restart the program. – supercat Jun 11 at 14:47
4

Example of using self-modifying code to save space and time: (or, actually, to overcome the limitations of the built-in interpreter)

In the C64 BASIC there is no way to evaluate string as formula (unlike, say, ZX Spectrum' VAL()). So in a function plotter program you have to either implement your own parsing and evaluation (which can be quite complex and inefficient in basic) or make your program incorporate the user-provided formula into itself, for example as a line similar to 1000 DEF FN F(X)=3*SIN(X/10)

4

Program code as data file

Microsoft Basic, e.g. on 8 bit Commodore machines, supported reading and writing program files on floppy disk like sequential data files.

OPEN 2,8,2,"EXAMPLE,P,W"

would open a file named EXAMPLE with type PRG for writing. Then one could create a program with a couple of PRINT# instructions, and CLOSE it, it'd show up as an executable file on disk. It should be written as it'd appear on the memory, i.e. line numbers as 16 bit integers, keywords should be tokenized etc.

Chain-loading or chaining

The LOAD instruction, when issued in a running program

999 LOAD "EXAMPLE",8

loads the named program from disk, replacing the running one, and starts executing it at the first instruction. As long as the loaded program is smaller than the first one, variables would be preserved, so they can communicate that way with each other.

Read-modify-write

A program could as well open the file containing its own code, read it one line at a time, modify some stuff on the fly, write it back to another file, then chain-load it.

4

The Sinclair QL updated with SMSQ/E and some BASIC extensions (all courtesy of about 1990) does that in a very straightforward way by using two multi-tasking BASIC jobs, one typing in code to the other:

100 EXEP "SBASIC" : REMark runs a daughter BASIC job
110 TYPE_IN "100 PRINT 'Hello, World'" & CHR$(10) : REMark TYPE_IN sends keystrokes from string to the current console
120 TYPE_IN "110 GO TO 10" & CHR$(10)
130 TYPE_IN "RUN" & CHR$(10)

This starts a second instance of BASIC, multitasking with the current one and "types in" a program, then runs it. Pretty much like in your hypothetical example.

3

The ZX Spectrum had the MERGE statement, which would read a saved program from a file and create or overwrite corresponding lines in the current program. (Any saved variables were also merged in the same manner.) However the program had to be in the ZX Spectrum's internal SAVE format, so you couldn't readily use it for arbitrary code. By default the program would load from tape, which made it inconvenient to use programmatically, but the Interface 1 device may at least have made it possible to use the MERGE statement as an overlay loader (I never had one though).

This doesn't directly count as it involves assembly language, but I wrote my own extensions to ZX Basic, one of which allowed the program to merge from a text file input stream instead. (As I recall you had to remember to include a GO TO statement in the text to resume code execution.)

2

When I came across an article about self modified Basic code on Mega65 project's blog. I wanted to try it myself and wrote the following code. The program itself doesn't make much sense but it works.

The trick is to place erroneous syntax in Basic memory. Then find the address of the erroneous tokens to replace it with whatever you want in runtime.

The code is written for CBM Basic 2.0 (C64) in petcat format.

10 gosub 9000 : rem find gosub address
20 print "{clear}{white}"
30 print "self modifyng basic"
40 ln=1000 : gosub 9100 : rem update address
50 gosub,12345
60 goto 50

999 rem ******************* subroutines
1000 print "this subroutine runs only once"
1010 ln=2000 : gosub 9100
1020 return

2000 print "this subroutine runs only once, too"
2010 ln=3000 : gosub 9100

3000 poke ja,44 : rem place comma again 
3010 end

8999 rem
9000 rem *********** find gosub address
9010 for ja=2048 to 40959
9020 if peek(ja-1)<>141 or peek(ja)<>44 then next
9030 return
9100 rem ****** change gosub address
9110 ln$=str$(ln)
9120 for i=0 to 5: poke ja+i,32 : next
9130 for i=1 to len(ln$): poke(ja+i),asc(right$(left$(ln$,i),1)) : next
9140 return

Blog post I've mentioned is in http://c65gs.blogspot.com/2018/04/gosub-variable-in-c64-basic.html

  • That certainly violate's the question's condition of not using any peek or poke. – Toby Speight Aug 5 at 9:49
1

One of the advanced techniques used on the TRS-80 was to build text strings containing character codes that represented Z80 machine language. These text strings were manipulated and combined and then executed from BASIC. Here is a typical sample from the popular game Android Nim:

9000 HA$=CHR$(160)+CHR$(181)+CHR$(186)+CHR$(144) 
9001 HB$=CHR$(128)+CHR$(143)+CHR$(182)+CHR$(185)+CHR$(143)+CHR$(128) 
9002 H1$=CHR$(128)+CHR$(143)+CHR$(183)+CHR$(187)+CHR$(143) 
9005 HC$=CHR$(160)+CHR$(186)+CHR$(176)+CHR$(149) 
9006 HD$=CHR$(128)+CHR$(143)+CHR$(189)+CHR$(179)+CHR$(142)+CHR$(128) 
9007 H2$=CHR$(128)+CHR$(143)+CHR$(191)+CHR$(179)+CHR$(143) 
9010 HE$=CHR$(160)+CHR$(176)+CHR$(181)+CHR$(149)

This would definitely qualify as self-modifying code. After running the program, if you did a LIST instruction, you would see strange characters where the code was modified.

0

If your willing to write your own interrupter then yes, but with specific limitation due to ram.

So lets say we have an AI.CMD file. Open,reading,writing, add lines, deleting line from any open file, were all part of the standard basic language.

Since we are talking about re-writing for an AI/self modifying program we are doing away with lengthy text descriptions.

General logic not fully executable code.

10 open #1,"AI.CMD"
20 input #1,VLINE,COMMAND$,PARAMETERS$
30 if AI(VLINE,COMMAND$,PARAMTERS$)<>0 then goto 20 (the AI decide not to run this line of code)
30 if COMMAND$="1" then print PARAMETERS$
40 if command$="2" then duplicate a different command

So every BASIC command would be assigned a number, and would get its own IF statement. So here the print command is represented as command #1. I am not saying this is going to be easy by any means.

Using the standard commands you can now modify the AI.CMD file.

Now deciding what to modify when is an incredible complex task.

As far as I know nobody actually did this.

How many programs or data files could be held at one time?

Generally, due to limited RAM, no files were held in memory at once, but instead a small pointer to said file was stored in memory. The limits depended on the version of BASIC, but you could have 8 files or more open.


Generally, your biggest problem is running out of memory to store the program. I started writing a TI Extended BASIC to assembly language convertor, but after getting even part way through I kept getting out of memory errors. The TI 99/4(A) only had 32K memory with a massive expansion box and memory card. So that ended that project.

The key here is the size and complexity of the self modifying routine would be very limited due to RAM size. Computers 1984 and before, didn't really have enough functionality to make attempting this practical.

0

Yes. In tokenized Basic interpreters (e.g. many early MicroSoft Basic derivatives) that include peek() function and poke statement, a running programs could examine its own tokens by peeking at memory, and modify the tokens on-the-fly by poking new values of tokens to be later executed.

The token formats for various statement and operations were reverse-engineered, with tables published and/or passed around various user groups.

This technique allowed, for instance, changing the expression A+B to A-B (etc.) for a larger but tight graphics loop without the overhead of an "IF" statement inside each loop iteration or repeating lots of code.

  • But the question specifically excludes using POKE! – Toby Speight Aug 5 at 9:51
0

Chipmunk Basic supports the merge command.

Running this program

10 merge "foo.bas"

with a file including this text (initially not in the Basic program):

20 print "hello 20"

will print

hello 20

Having a program write a .bas file before using the merge command on it allows even more modification.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.