Simple answer: Because it's the way BASIC is defined.
Original BASIC had no way to access files at all. DATA lines were the only way to add predefined data to a program. The idea was essentially to have a stack of items - originally only numbers, strings where only added way later - that could be read like a stack of cards.
More Important: It is already close to an optimal approach within the limits set by BASIC.
Isn't this rather wasteful on an 8-bit computer with perhaps anywhere from 4KB to 48KB of RAM?
In what way wasteful? It's in fact less wasteful than any other way. Keep in mind that data that needs to be used need to be stored in source, so everything has to be present in source code anyway.
A Basic Issue about BASIC
It's important to remember that BASIC, at least for most micro computer implementations, is a source code interpreter (*1). The source code is not compiled into machine code, but is the 'code' to be executed. And this source code is to be preserved (*2).
The imperative of 100% retrieval of original source text does restrict abilities to reorganize.
Some Number Crunching:
A data line consists of
- line overhead,
- the DATA instruction (token),
- the data items themself and
- delimiters inbetween
Whatever form a more 'space savvy' may have, I guess we can agree that it needs at least to include the data items as written and delimiters. In addition such a form must of course be placed in a line, thus having line overhead as well and at least an instruction.
In the end exactly exactly the same minimum space requirement as for DATA lines. So storage in terms of source text can not optimized a lot.
Except, there is of course the loading routine, moving the data into variable arrays. In most cases a simple FOR/NEXT loop iterating over the data array. While being some code, it isn't a lot thereof:
10 FOR I=0 TO 20
20 READ A(I)
30 NEXT I
In MS-like BASICs that'S about 14 bytes plus line headers not really a big overhead. Any other method may need more. As it still must name the array to be filled plus optional offset to start and so on.
Using Direct Assignment
Alternatively, why wasn't it particularly common for programmers to assign array items directly — as this Apple 1 port of ELIZA does — to leave more RAM free?
Because it eats up again more data? Keep in mind, BASIC is a source level language, so each and every assignment will exist in Source and (after being executed) in data storage. Instead of having one item consisting of just its source representation, it now needs to be preceded by an additional variable name and an assignment operator. This adds up a lot.
The reason why the Apple 1 Version uses this is rather obvious: There was no DATA statement.
Using Multiple Assignments
Cool idea (and borrowed from BASICS parent FORTRAN). This could for example be done by introducing a multi assignment statement. For example like
would load the letters A..F into A(1..6). Looks great, but needs more more storage, as now each line also needs the target named - after 3-4 lines this may eat up more source space than a corresponding FOR loop.
In addition this would remove the ability to structure data lines in a meaningful way, making them even harder to read. Just think a structure of corresponding items, like an address consisting of zip code, city, street and number. With classic data items this will look like
10 FOR I=1 TO 2
20 READ ZP(I), CY$(I), NR(I), ST$(I)
30 NEXT I
100 DATA 81739, "München", 34, "Waldheimplatz"
110 DATA 81541, "München", 6, "Schweiger Straße"
With multiple assignment it'll be more like
10 LET ZP(1)=81739, 81541
20 LET CY$(1)="München", "München"
30 LET NR(1)=34, 6
40 LET NR(1)="Waldheimplatz", "Schweiger Straße"
Which one is more pleasant to read? Not to mention that all these cryptic data lines now have to be at the begin of the program (or need a subroutine to be moved back).
While it would be rather useful as well in situations outside of replacing DATA statements (*3), this would not only need more source code with lower readability (*4) but still have the internal representation (in data storage) in addition to source data.
So far the use of DATA statements seems the most space saving, but what about
A 'Special' Approach do DATA Data
One may of course think of a way to handle DATA statements like before, but on assignment only 'copy' a pointer to the data chunks within the DATA line. Could be a nice idea, but it comes with a hefty cost in code (*5). Now variable handling needs to distinguish between pointers into DATA statements and such into data storage and to handle copy on modification (*6)
Also, this would only make sense with strings, as data, including numeric is stored in DATA statements as ASCII (to be able to recreate source code). Converting them during each access from ASCII is less than great for performance, so they better get converted just once.
DATA is Already an Array
But what inspired implementers of the various BASICs to build in a DATA/READ feature instead of some RAM-friendlier mechanism like a syntax for defining read-only arrays?
DATA is already a read only array. It's a one dimensional array, starting with the first item, accessed sequentiallyvia consecutive READs. Thanks to RESTORE (*7) the read pointer can be reset any time to parse the data stack again. Not fast - in fact extreme slow - but the most space saving handling :)
Long Story Short
- DATA statements area way to handle a static input 'file' within the program.
- There is no more efficient way than the existing within the context of BASIC as interpreter (and the restricted capacity of 8 bit systems).
- As soon as a system supports reading from data files (*8) that should be used (*9).
*1 - Funny considering the original BASIC being a compiler :))
*2 - Many BASICs employ methods to reduce storage need by using crunching methods like tokenization of keywords, some use (like Sinclair) use optimized constant storage to speed up execution (but increasing space), but all handle the code in a way that the original text can be reconstructed down to the last dot and comma (e.g. when listed).
*3 - Something I'd really would welcome in a new BASIC.
*4 - Well, one could as well think of a multiple assignment with multiple variables as well on the left side, but I'd feel that would quite leave the path of BASIC. Not at least as it would need quite complex code, as until now there is always only a single left hand variable.
*5 - Another idea for a more modern BASIC where there is room for more code.
*6 - As MS does already for the ASCII content of strings - See Supercat's answer
*7 - Introduced with Third Edition BASIC in 1966
*8 - Introduced with 5th Edition BASIC in 1970.
*9 - The overhead consists of the same FOR loop plus opening and closing the file. In fact, most BASICs use INPUT for reading with a syntax quite like the READ instruction, so it really ends in moving the data lines to a file, removing lien numbers and the DATA statement and saving it. Quite what I call easy upward compatibility.
RECALLstatements for loading variables from cassette tape. Perhaps they figured that, if you had a lot of data, that was the best way to handle it. There are OS equivalents in ProDOS BASIC.System for loading variables from disk. Some programs just PEEKed values out of a binary blob in memory.