How did software engineers test their code in 19xx? [closed]

Question

There is a plethora of tools available for engineers to test their code these days. However, frameworks like Mocha.js came around only in 2011 with the ascent of NPM. So I wonder what were the practices around software design a few decades ago?

Answer 1

Back when I started in the late 1980's, we still had test suites and test cases. It's just that all the tests were carried out by hand. Unit tests were pretty much unheard of - the developer just did some ad hoc testing before checking his (it usually was a he) code in. Then there was formal testing that was structured into several tiers.

You had your module tests, perhaps integration tests, functional tests, system tests and finally (you hoped) acceptance tests. Sometimes there were also user tests, but showing the system to the users before it was ready was sometimes considered politically sensitive. One system in particular stands out. I worked on the first British computerised passport issuing system. The office staff who were then still writing out passports by hand, were kept completely out of the loop during development because it was feared they'd all go on strike or something. In the end, they did anyway and, as a result of not consulting the users, basic mistakes were made in design of the user interface.

Anyway, on that particular project, we designed the individual modules using a technique called JSP (Jackson Structured Programming). We drew our JSP diagrams and then we converted that into a special kind of pseudo code. We'd use the JSP diagram to enumerate all of the possible paths through the code and produce a series of test scripts from that. Then (and this is the first step done not with pen and paper) we typed the pseudo code as comments in an empty Cobol source file and wrote the code between the comments. Then we would run the test scripts.

The other types of tests were made by reference to a test plan and the functional specification. There would, for example, be a section in the functional specification that described the facility to enter a new passport application from an application form. You would examine that and write a series of test cases - probably filling in real application forms - and then you'd create a test script for the tester to follow (note: all of this is being done by hand, you wrote the scripts out with pen and paper and sent them to the typing pool to be typed). An army of testers would then run the tests and raise defect reports for when they went wrong.

The whole process from start to finish was unbelievably tedious, especially the module tests, which meant, of course, that they were frequently not done as well as they might be. Hence, it was not unusual to find fairly basic bugs even as late as acceptance testing. Once the system was delivered and in support, regression testing was an unattainable fantasy. The support team simply didn't have the resources to run through the mountain of paper rerunning the functional tests.

For those who are interested, the above system was written for Tandem Non Stop Cyclone systems which were designed with redundant hardware for maximum uptime. One of the system tests involved physically pulling a CPU module out of the rack while the system was running to simulate a hardware failure. The software was mostly written in Cobol talking to ISAM files at the back end. It ran in a transaction process monitor and the UI was written in a Cobol like screen definition language whose name I forget.

Answer 2

By late 1983, Apple developers had developed an almost complete Apple Macintosh OS and were working on software for it.

MacWrite and MacPaint were pretty much feature complete but still needed a lot of testing, especially in low memory conditions. MacPaint needed to allocate three off-screen buffers, with each the size of the entire screen, so it was always skirting the edge of running out of memory, especially when you brought up a desk accessory, but the specific sequences that led to crashes were difficult to reproduce.

This lack of memory meant that there was not enough memory for a large testing program, nor a large set of test-cases. Steve Capps, whilst working on the Guided Tour disk, realised that the underlying system that powered the playback of pre-recorded actions could be used as the basis for one of the first GUI fuzzers: the Monkey.

The Monkey was a small desk accessory that used the journaling hooks to feed random events to the current application, so the Macintosh seemed to be operated by an incredibly fast, somewhat angry monkey, banging away at the mouse and keyboard, generating clicks and drags at random positions with wild abandon. It had great potential as a testing tool, so Capps refined it to generate more semantically rich events, with a certain percentage of the events as menu commands, a certain percentage as window drags, etc.

For more information and the source of these quotations, see Folklore.org: Monkey Lives.

Answer 3

Admittedly, your question is way too broad - Even today, that question cannot be answered for the bandwidth of what commonly qualifies as "software". Embedded Software requires (and required) an entirely different test strategy and, obviously, the approach to testing safety-critical software like a moon lander will be entirely different from that of a home computer game.

When I started developing software in 1986, I was working on a satellite payload system for telemetry and range finding. That system was entirely developed on IBM PCs that had a number of Z-80 daughter boards (the target system was Z80-based). The entire system was module tested on the PC that emulated the on-board connectivity. There was only one single "real" target system that was heavily guarded for safety and confidentiality issues and only accessible after a long-winding procedure after passing quite a number of (physical) gates. That system was normally used only for integration and acceptance tests. What you did was feed your software running on the Z80 boards with proper stimuli from the PC using small (PC) assembler programs and hope you'd get the right results out of the simulated base station comms. If something didn't work, you'd start a CP/M debugger to remotely debug the Z80 software on the boards. You didn't have any symbol support or even symbolic debugging, but rather worked with large piles of paper from the linker map file and assembler source code, single-stepping instruction by instruction. Paper was generally much more important to programming than it is today. The teams were also much smaller that they probably are today - On that whole system, only 4 developers were working for about a year. As there was no way to on-board-update the software in case you found an error after launch (or a simple way to recall the system for a quick factory fix... Anything that was invented by then would have involved a crater...), the system had a number of very space-limited "patch areas" (empty static RAM pages that were kept as reserve to be able to replace faulty software with small "backpack" routines) and testing was really nearly half of the development effort if I remember right.

On another job some years later, we were working on technical GUI software - There was no such thing as automated testing for GUIs, even less than on the satellite payload - All testing would be done by hand (and mouse!) according to "user stories" (what a modern term...) that were documented in huge piles of paper (again) with expected input or output. After such a story, a database script was run to see whether the database contents looked like expected, and that was the only "automated testing" we had. We also didn't have a dedicated test team - for the bi-yearly product releases, development was simply shut down for a month or so and all hands were called in to do a number of incremental test sessions, document the results (again on paper) and then fixed the bugs you found. At least, nobody was normally assigned to test something they had not written themselves, so you had the chance to see parts of the software you weren't aware they were existing before that.

In another job, I was working on the control software for large spinning machines - We had a cut-down "real machine" (just cut down in size so that each sensor/actor/motor on the real thing was actually present at least once) in the middle of the office that could be (and was) used for testing. The software was developed on OS/9 development systems (PCs with 68k cards, embedded systems and some Atari STs (!) running OS/9) and then tested on the machine. Main testing was, however, done by finding out whether the various sub-systems that were communicating with each other over RS-232 lines exchanged the expected messages - so your most important testing tool was the protocol analyzer.

Answer 4

IIRC, there were conformance tests (input decks with known outputs) back in the day of FORTRAN programs on punched cards.

Answer 5

Agree with tofro, a broad question with no simple answer and depends on how critical the application was and the budget.

c1982, writing assembler (2650) for a TV video games console, if the game could be played and no-one could crash it, make it do something odd, or get strange displays or sounds, then it was good to go. For the most part if anything at all was amiss the output would be total garbage and easy to spot, but there wasn't much technology available to find the cause.

c1985, again writing assembler but for a system with 12 6809 processors. Algorithms were developed in psuedo-code, then I wrote and thoroughly tested code for each psuedo-code structure (ie IfThenElse, ForN=x->y etc) before converting the algorithms to code. This unit fed data to a minicomputer, and the guy who was writing the code for that created test beds for the unit. These two components were part of a much larger communications loop, but I never got any feedback from system integration and field testing.