18

Take for example this BASIC version of ELIZA which starts out (in lines 50–170) by a number of READ loops which copy DATA (lines 1340 and following) into a handful of arrays.

Isn't this rather wasteful on an 8-bit computer with perhaps anywhere from 4KB to 48KB of RAM?

That is, first the program itself, including all its lines of DATA statements must be loaded into RAM. Then when the program runs it proceeds to DIM additional RAM and READ it into a separate "working copy".

Now I suppose if that "working copy" needed to be modified, this data duplication would be useful to get a fresh copy that reset to its initial values on each RUN. But what inspired implementers of the various BASICs to build in a DATA/READ feature instead of some RAM-friendlier mechanism like a syntax for defining read-only arrays?

Alternatively, why wasn't it particularly common for programmers to assign array items directly — as this Apple 1 port of ELIZA does — to leave more RAM free?

7
  • 1
    Applesoft has the STORE and RECALL statements for loading variables from cassette tape. Perhaps they figured that, if you had a lot of data, that was the best way to handle it. There are OS equivalents in ProDOS BASIC.System for loading variables from disk. Some programs just PEEKed values out of a binary blob in memory. – fadden Jun 23 at 0:06
  • It actually makes me think of the modern "data-driven" paradigm, with BASIC not necessarily having any access to a DOS or a data file - so, it's in the code. en.wikipedia.org/wiki/Data-driven_programming – Brian H Jun 23 at 13:45
  • 2
    BASIC wasn't designed for tiny microcomputers, it was initially run on Honeywell mainframes. – Barmar Jun 23 at 21:12
  • 4
    How do you think this could be memory optimized? Apart from loading constants from disk, they will always have two instances where they need to live: one in the program code, the other in the actual variable - And that's true for all interpreted languages. The BASIC for Sinclair computers made this a bit easier: You could save variable values together with the program - Thus, you assigned whatever you wanted in the arrays, then deleted that code and saved the program - the initial values where already set when the program was loaded again. – tofro Jun 24 at 12:02
  • @tofro: What you describe may be one approach that would fit what the author had in mind. If there were a statement to dimension a "permanent array" which would not be cleared when a program is edited or executed, and which would be saved with a program, that could eliminate the need for "data" statements in many cases. One could near the end of a program have some code to allow a user to edit the contents of a permanent array, and then have a programmer use that to put data into the array without bothering with DATA statements. – supercat Jun 24 at 16:36

10 Answers 10

19

DATA is a kind of weird holdover from FORTRAN and batch-processing. It is not memory efficient, but it was part of the programming vocabulary of the day.

In FORTRAN, inline data can look like this:

  DIMENSION MARKS(10)
  DATA MARKS/31,32,33,34,35,36,37,38,39,40/

This fills the ten integer values of the MARKS array with [31,32,33,34,35,36,37,38,39,40] respectively. Since FORTRAN is compiled, there isn't any storage overhead. This is aside from run-time data parsing, where data cards were read by the FORTRAN program as it ran.

Dartmouth BASIC may have had less of an overhead using DATA statements, as it was dynamically compiled on the client-facing machine, and submitted to the actual processor as if it were a series of batch jobs. Most BASIC interpreters since the 8-bit days don't use this model, so DATA just sits about taking up memory.

You can eliminate redundant data storage by saving it to a separate (binary?) data file, but that requires file I/O and storage hardware. This isn't always evenly defined for systems running BASIC.

I know of only one interpreter that allows for inline array declaration, but there are likely others. BBC BASIC (since V5, so not on the 8-bit versions) allows you to do this:

10 DIM A(10)
20 A()=31,32,33,34,35,36,37,38,39,40
30 PRINT A(2)

This still has some redundant storage, but doesn't need code inside the program to interpret the data. It should be noted that — unusually for BASIC — this method fills the array from A(0) to A(9) rather than from A(1) to A(10). So A(2) here is equal to 33.

2
  • 2
    The modern equivalent is probably the "here document", e.g. in shell scripts. Sometimes you just need to carry the data with the program, whether that's in the same tape or the same file. – pjc50 Jun 23 at 8:13
  • Pretty much, but with shell scripts "I am now my own stdin" is where things merely start to get weird. PostScript was another language where code and data were mixed in one stream, as often ran as a serial spool from a host. I tell ya, it didn't make debugging a multi-GB postscript job easier when the platemakers were screaming for the next job ... – scruss Jun 23 at 16:54
13

D'oh! I can provide at least a partial answer to my own question:

Alternatively, why wasn't it particularly common for programmers to assign array items directly — as this Apple 1 port of ELIZA does — to leave more RAM free?

What I was imagining here was that instead of something like:

 5 DIM A$(4)
10 FOR I = 1 TO 4 : READ A$(I) : NEXT I
20 DATA "ONE", "TWO"
30 DATA "THREE", "FOUR"

the programmer might "optimize" as:

10 DIM A$(4)
20 A$(1) = "ONE" : A$(2) = "TWO"
30 A$(3) = "THREE" : A$(4) = "FOUR"

so as to avoid having an "extra copy" from all the DATA statements. But of course this still has two copies of the data: one in the source code and one in A$().

So apart from some sort of language mechanism for referencing constant data, the READ/DATA combination is a bit less tedious than individually assigning each array index, and probably uses less RAM for storing the source code.

(The Apple 1 example accomplishes two distinct things: for one, the original Integer BASIC didn't include DATA/READ so the author had to go without regardless. And for two, it is somewhat optimized in that rather than DIMensioning full lists for all, some variables like R$ and K$ get assigned to a single value at a time and a computed GOSUB is used to "index" into the available values instead.)

6
  • I don't know how all BASICs implemented it, but hopefully for something like READ A$: DATA "TEST" - the internal pointer for A$ likely points to the bytes in the DATA statement program text until something modifies it - same for string literals in code such as A$="whatever". Of course for numerics this doesn't work. – LawrenceC Jun 23 at 11:35
  • 2
    @LawrenceC I cannot say for sure wrt READ, but pointer optimization tricks for strings were generally a later development and not evident for 8-bit BASIC interpreters. Virtually all string operations copied the string. The only sure exceptions I can think of are 1) parameter passing to a subroutine (which varied by implementation) and the LHS version of the MID$ function. Because of the last one, it would not have been safe to use a pointer into DATA strings, unless some fancy tracking was used. – RBarryYoung Jun 23 at 13:11
  • 1
    Atari 8-bit BASIC (admittedly an odd duck) calls READ/DATA/RESTORE "Internal I/O" due to how it treats DATA as an input source: "XINPUT handles the assignment of the DATA values to the variables" archive.org/details/ataribooks-the-atari-basic-source-book/page/… – Jim Nelson Jun 23 at 18:55
  • Not only that, if the array is large (over a hundred entries, like ELIZA, rather than three or four), all of those assignment statements will take measurably more code space than the equivalent DATA plus a read loop. – hobbs Jun 23 at 19:44
  • Perhaps noteworthy that the textual representation of data that is not strings differs from the binary one (e.g. the text bytes 54-53-53-51-53 correspond to the 16 bit integer 255-255). The data is "duplicated" by necessity. To implement an extra hoop for strings where both source code and internal representation coincidentally are the same would have been difficult, not least space-wise. – Peter - Reinstate Monica Jun 24 at 15:47
11

READ and DATA were features of Dartmouth BASIC in 1964. Thus, anything that wanted to call itself BASIC really had to implement them.

The computing model of the day was "read some input; do some computing; write some output".

Normally, in other languages on other contemporary systems, you'd be using card or paper tape. The data were on cards or tape, probably immediately following the source program.

(An online multiaccess system that I used followed that model for Algol 60 programs, even though - luxury! - we had disc files for storage, rather than paper or cardboard. But the data could still be present in the same file as the source code, after the final end),

BASIC DATA follows that same structure, except that organization by line number means DATA can occur anywhere. But logically, it's "an input data file".

With the benefit of hindsight, I think DATA can achieve a useful separation between the program and the data that the program operates on, in a way that would not be possible if explicit assignment statements were used.

I'd say the ELIZA program followed that model so that the vocabulary could be tinkered with, without modifying code.

And, of course, consider the case where not all DATA are consumed at once.

10   READ A
20   IF A < 0 THEN 90
30   REM COMPUTE SOMETHING WITH A
40   ...
50   ...
80   GOTO 10
90   END

This is the difference between 'initialization' and actual data to be processed by this run of the program.

Writing a block of DATA with 100 values to be used consecutively is more convenient, and easier to update, than assigning 100 array elements.

8

TL;DR:

Simple answer: Because it's the way BASIC is defined.

Original BASIC had no way to access files at all. DATA lines were the only way to add predefined data to a program. The idea was essentially to have a stack of items - originally only numbers, strings where only added way later - that could be read like a stack of cards.

More Important: It is already close to an optimal approach within the limits set by BASIC.


In Detail:

Isn't this rather wasteful on an 8-bit computer with perhaps anywhere from 4KB to 48KB of RAM?

In what way wasteful? It's in fact less wasteful than any other way. Keep in mind that data that needs to be used need to be stored in source, so everything has to be present in source code anyway.

A Basic Issue about BASIC

It's important to remember that BASIC, at least for most micro computer implementations, is a source code interpreter (*1). The source code is not compiled into machine code, but is the 'code' to be executed. And this source code is to be preserved (*2).

The imperative of 100% retrieval of original source text does restrict abilities to reorganize.

Some Number Crunching:

A data line consists of

  • line overhead,
  • the DATA instruction (token),
  • the data items themself and
  • delimiters inbetween

Whatever form a more 'space savvy' may have, I guess we can agree that it needs at least to include the data items as written and delimiters. In addition such a form must of course be placed in a line, thus having line overhead as well and at least an instruction.

In the end exactly exactly the same minimum space requirement as for DATA lines. So storage in terms of source text can not optimized a lot.

Except, there is of course the loading routine, moving the data into variable arrays. In most cases a simple FOR/NEXT loop iterating over the data array. While being some code, it isn't a lot thereof:

10 FOR I=0 TO 20
20 READ A(I)
30 NEXT I

In MS-like BASICs that'S about 14 bytes plus line headers not really a big overhead. Any other method may need more. As it still must name the array to be filled plus optional offset to start and so on.

Using Direct Assignment

Alternatively, why wasn't it particularly common for programmers to assign array items directly — as this Apple 1 port of ELIZA does — to leave more RAM free?

Because it eats up again more data? Keep in mind, BASIC is a source level language, so each and every assignment will exist in Source and (after being executed) in data storage. Instead of having one item consisting of just its source representation, it now needs to be preceded by an additional variable name and an assignment operator. This adds up a lot.

The reason why the Apple 1 Version uses this is rather obvious: There was no DATA statement.

Using Multiple Assignments

Cool idea (and borrowed from BASICS parent FORTRAN). This could for example be done by introducing a multi assignment statement. For example like

LET A$(1)="A","B","C","D","E","F"

would load the letters A..F into A(1..6). Looks great, but needs more more storage, as now each line also needs the target named - after 3-4 lines this may eat up more source space than a corresponding FOR loop.

In addition this would remove the ability to structure data lines in a meaningful way, making them even harder to read. Just think a structure of corresponding items, like an address consisting of zip code, city, street and number. With classic data items this will look like

10 FOR I=1 TO 2
20 READ ZP(I), CY$(I), NR(I), ST$(I)
30 NEXT I
100 DATA  81739, "München", 34, "Waldheimplatz"
110 DATA  81541, "München", 6, "Schweiger Straße"

With multiple assignment it'll be more like

10 LET ZP(1)=81739, 81541
20 LET CY$(1)="München", "München"
30 LET NR(1)=34, 6
40 LET NR(1)="Waldheimplatz", "Schweiger Straße"

Which one is more pleasant to read? Not to mention that all these cryptic data lines now have to be at the begin of the program (or need a subroutine to be moved back).

While it would be rather useful as well in situations outside of replacing DATA statements (*3), this would not only need more source code with lower readability (*4) but still have the internal representation (in data storage) in addition to source data.

So far the use of DATA statements seems the most space saving, but what about

A 'Special' Approach do DATA Data

One may of course think of a way to handle DATA statements like before, but on assignment only 'copy' a pointer to the data chunks within the DATA line. Could be a nice idea, but it comes with a hefty cost in code (*5). Now variable handling needs to distinguish between pointers into DATA statements and such into data storage and to handle copy on modification (*6)

Also, this would only make sense with strings, as data, including numeric is stored in DATA statements as ASCII (to be able to recreate source code). Converting them during each access from ASCII is less than great for performance, so they better get converted just once.

DATA is Already an Array

But what inspired implementers of the various BASICs to build in a DATA/READ feature instead of some RAM-friendlier mechanism like a syntax for defining read-only arrays?

DATA is already a read only array. It's a one dimensional array, starting with the first item, accessed sequentiallyvia consecutive READs. Thanks to RESTORE (*7) the read pointer can be reset any time to parse the data stack again. Not fast - in fact extreme slow - but the most space saving handling :)


Long Story Short

  • DATA statements area way to handle a static input 'file' within the program.
  • There is no more efficient way than the existing within the context of BASIC as interpreter (and the restricted capacity of 8 bit systems).
  • As soon as a system supports reading from data files (*8) that should be used (*9).

*1 - Funny considering the original BASIC being a compiler :))

*2 - Many BASICs employ methods to reduce storage need by using crunching methods like tokenization of keywords, some use (like Sinclair) use optimized constant storage to speed up execution (but increasing space), but all handle the code in a way that the original text can be reconstructed down to the last dot and comma (e.g. when listed).

*3 - Something I'd really would welcome in a new BASIC.

*4 - Well, one could as well think of a multiple assignment with multiple variables as well on the left side, but I'd feel that would quite leave the path of BASIC. Not at least as it would need quite complex code, as until now there is always only a single left hand variable.

*5 - Another idea for a more modern BASIC where there is room for more code.

*6 - As MS does already for the ASCII content of strings - See Supercat's answer

*7 - Introduced with Third Edition BASIC in 1966

*8 - Introduced with 5th Edition BASIC in 1970.

*9 - The overhead consists of the same FOR loop plus opening and closing the file. In fact, most BASICs use INPUT for reading with a syntax quite like the READ instruction, so it really ends in moving the data lines to a file, removing lien numbers and the DATA statement and saving it. Quite what I call easy upward compatibility.

1
3

Simply, because most BASIC programs weren't sufficiently memory-constrained to need to do anything differently, and reading everything into memory made things simpler.

There are actually some BASIC programs that would not read all of DATA into an array, but rather just READ a single item (or small record of related items) that needed processing right now. (Typically these would be relatively large games, rather than other kinds of software.)

But this requires an additional statement (RESTORE) -- in modern BASIC this accepts an expression specifying the specific line number at which the next READ will start reading, allowing the program to use the DATA itself as a kind of indexable array.

Some older versions of BASIC did not allow specifying a line number in RESTORE, or only allowed a constant value rather than an expression. In these cases, to accomplish the same thing the program would need to RESTORE to the start of DATA (or a fixed location) and then READ and discard values in a loop until reaching the one that they actually wanted. And again, some programs did actually do this. (And even older versions of BASIC lacked the RESTORE statement at all.)

It should be obvious that both of these alternatives come at a performance cost vs. simply having all the data in a real array, so there is little incentive to do this unless you start running into memory limitations.

2

String variables and arrays store the length and address of the text. If a program computes a string value, then it will need to construct a string in memory which is filled from the top downward, but at least in Microsoft-derived BASIC for the 6502 and probably 8080, when a program performs a READ A$, the interpreter will store the address of the appropriate part of the DATA statement, rather than making a new copy of the string text.

That having been said, the design of DATA statements is pretty horrid for numeric information, both from a memory usage and execution time standpoint. I find it curious that back in the day people didn't routinely use a read/data loop to construct a machine-code routine that could process things much more efficiently (so that e.g. x=usr(12345),"ABCD1234" would store four bytes with values 0xAB 0xCD 0x12 0x34 to addresses starting at 12345.

1

The only alternative aproach to DATA is to assign initial values to initialized variables either directly (which doesn't save anything, the data is still present twice, both in the variables area and in the program code), load it from a binary file directly into the variable (a possibility on Apple BASIC, for example), or store initialized data together with the program on, for example, tape).

The last option was chosen by the designers of Sinclair Basic for the ZX 81 and ZX Spectrum computers: When you save a program, these computers will also save and restore the current contents of the variable area, which eliminates the need for DATA, READ and RESTORE completely. Typical space-saving programs would enter variable values either through direct commands or small "initializer" programs, then delete all program lines and enter the actual program that could then access the initialized values. SAVEing that new program would also save the initialised variables together with that, allowing the program to use these initialized values, which only existed once in memory now (as long as you didn't use CLEAR or otherwise change the values)

2
  • Sure, these are all alternative, work arrounds and alike (while missing the way of reading a data file in addition) but not really why there is DATA and working how it does in the first place, isn't it? – Raffzahn Jun 24 at 18:56
  • @Raffzahn The question is not, IMHO "why there is DATA?" or "how does it work?", but rather "why does it waste memory?". – tofro Jun 25 at 7:18
1

An additional workaround I've seen, in the book Writing Basic Adventure Programs For The TRS-80 went into this specifically; in an attempt to write a simple adventure-type game but stay within the restricted memory limits, the author recommended putting information into DATA blocks at the end of the code and then never reading it into variables, but just read it out directly using PEEK statements to find where it was stored in memory. It was really quite ingenious.

And as mentioned elsewhere, you could just populate your arrays and then BSAVE that area of memory into a binary file, and BLOAD it back in, which was the strategy that, for example, the game Odyssey used.

0

I think that if this turned into a problem, it could be mitigated with hacks. Some BASICs had commands for deleting a range of lines; that could be used to blow off the DATA section of the program. Even if that is not available, or doesn't liberate the memory in the right way, it could be hacked.

In developing a BBS on an Apple II machine, I used an overlay technique to fit a larger program into memory.

When the sysop (system operator) entered the administrative area of the BBS software, it loaded all that code from disk, and replaced a range of lines of the program with that code. Upon returning from the admin menus to the regular BBS area, it would load that again and discard the admin code.

I don't remember all the details, but I remember it involved a machine language routine which walked the linked list of the program's lines and did some splicing. It could have been that the loaded part of the program overwrote the memory starting at the right address, and then the machine language routine just repaired the linked list of the program.

The main point is that the BASIC program is data, making it possible to surgically remove some unneeded part of it, similarly in spirit to the way the Linux kernel discards functions only used for initialization. It may involve code that is specific to the BASIC implementation, depending on the details of the program's representation in memory and the memory layout.

In this particular situation, with regard to the DATA statements, we have to be careful. If the READ is implemented such that it's leaving the string data in place, and having the variables or array elements pointing to it, we cannot just remove that part of the program, because we end up with dangling pointers. (And in that case, we have less reason to anyway). We need to perform some string operation that will cause the string to be moved into the heap.

0

Basic programs "tended" to use Data and Read statements because these statements were defined parts of both the original Dartmouth implementation, and documented in the ANSI/ISO Minimal Basic and Full Basic Standards.

Given the generic mainframe computing paradigm of separate "program" and non-interactive "input data", one could concatenate together these two parts when feeding a Basic implementation, as long as care was taken to use separated ranges of line numbers for the 2 text files.

As for the "waste", this is an implementation detail, as there is no reason for a (non-self modifying) Basic implementation to read the full text of a program's text into memory. Applesoft and other *soft variants tokenized the actual Basic text, instead of storing the actual text. Other Basic implementation compiled the source into machine code, and ignored Rem statements. In other hypothetical implementations, the Data statement's contents could be compressed, or any Data statements could also be considered as just a separate file resource, and not read of off program storage media until actually needed by Read statements at run-time.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.