17

Prior to 1972, was there any automated unit-testing? Did any assemblers include a testing a framework?

I can find some academic journals on this topic from the late 70s -- most of which I don't have access too, but I don't see any implementations or examples of it. What providers pushed automated unit testing? Did automated software unit testing start with tooling for C or are there prior examples?

  • 9
    Unit testing wasn't a thing until 3 decades after C. It has nothing to do with language. – user3528438 May 7 at 16:36
  • 9
    @user3528438 don't believe that was true, ieeexplore.ieee.org/abstract/document/1646913 "Typical testing activities may involve many hundreds of tests. An automatic software test driver assists the tester by managing all of the test data, and automatically running the tests. Savings during regression testing can be significant." - 1978 – Evan Carroll May 7 at 16:45
  • 3
    I find your wording confusing. Are you asking if there were unit test frameworks available for assembly languages prior to a certain date? – Jim Nelson May 7 at 17:32
  • 16
    There definitely needs to be a definition of "unit testing" for the question to be answerable. The notion of testing individual "units" of something before doing an integration test predates the concept of software. You can probably find industrial process literature from the 1800's that lays out the notion quite clearly, and ancient texts that cover more or less the same basic concepts when talking about making parts of bronze swords or whatever. – wrosecrans May 7 at 23:42
  • 4
    I don't think you'll find much pre-dating mid-1980s work at IBM, as far as trying to standardize a process or tooling. Before that was bespoke solutions per project and academic flim-flam. – Brian H May 8 at 0:55
33

Yes: The Apollo Guidance Computer

Uodate: Computers in Spaceflight: The NASA Experience confirms that unit testing was performed on the Apollo software:

The simulations followed individual unit tests and integrated tests of portions of the software. At first, MIT left these tests to the programmers to be done on an informal basis. It was very difficult at first to get the Instrumentation Laboratory to supply test plans to NASA. [...] NASA exerted some pressure on MIT to be more consistent in testing, and it eventually adopted a four-level test structure based largely on the verification of the Gemini Mission Control Center developed by IBM in 1964. This is important because formal release of the program for rope manufacture was dependent on the digital simulations only.

p. 47

At one point Bill Tindall, NASA's overseer of the MIT effort, complained about the lack of unit testing:

On June 13, Tindall reported that the AS-204 program undergoing integrated tests had bugs in every module. Some had not been unit tested prior to being integrated. This was a serious breach of software engineering practice.

p. 50

Tindall's memo is archived here. You can see that he calls it a "unit test" on June 13, 1966:

In fact, some of the programs –- the 30 and 40 series -– have not even been unit tested, although they have been included in the program assembly currently being used for system tests. This is certainly a very unsatisfactory situation.


The command modules and lunar modules of the Apollo space program were each controlled by the same model of guidance computer. It was the first computer built from integrated circuits (NOR gates).

The software for the AGC was written by a team led by Margaret Hamilton at the Draper Laboratory of MIT. It was written in assembly language, with an assembler written by the Draper team.

Hamilton was particularly concerned about how software handled errors. At the time, errors were often dismissed as "bugs". Hamilton knew that this was not acceptable for the Apollo software, not only because the executable code was hand-woven into core memory -- making changes difficult -- but also because the lives of the astronauts were at stake.

Individual sections of the software were tested as they were written. Special tests were made on the running code:

Executable tests may have first been introduced by Margaret Hamilton on the Apollo project in the mid-1960s, where she originated a type of executable checking that we now call “static code analysis.” She called it “higher-order software,” by which she meant software that operates against other software rather than directly against the problem domain. Her higher-order software examined source code to look for patterns that were known to lead to integration issues.

https://www.leadingagile.com/2018/10/unit-testing-shell-scriptspart-one/

Hamilton named their method "software engineering". She later formalized it, wrote over 130 papers, and started several companies specializing in software engineering.

In 1986 she began her current company Hamilton Technologies. At Hamilton Technologies projects focus on work Hamilton began while at MIT including a Universal Language System. This system develops software off the theory “Development Before the Fact.” It focuses on the function and timing of data within a system to prevent errors before they occur. The automation of this system created the 001 Tool Suite (pronounced double-oh-one), an updated version of the work Hamilton began in her first company. The 001 Tool Suite is used to develop automatic software code and simulations.

https://sheromargarethamilton.wordpress.com/

In 2016, Hamilton received the highest civilian honor in the U.S. -- the Presidential Medal of Freedom -- for her groundbreaking work in software engineering.

| improve this answer | |
  • 4
    Static code analysis (lint) is not unit testing. It also isn't "executable checking" so the quote above doesn't make sense to me. – snips-n-snails May 8 at 5:15
  • 2
    If it is running it is a unit test. If there is a repeatable test without manual intervention it is automated. Question answered. I am sure the V2 Rocket program would have something in the way of a harness for gyro testing, if you consider that guidance system a computer. – mckenzm May 8 at 5:54
  • 4
    @mckenzm if it's running the code, then it is not a static test. If it's a repeatable automated process that runs on the code (as opposed to executing the code), then it's not a unit test. "examined source code to look for patterns" is code verification and linting, which are good and valuable things, but not related to unit testing. The question is about about one type of automated testing, the sections quoted here are about a different type of automated testing. – Peteris May 8 at 14:16
  • 3
    The statement "a type of executable checking that we now call “static code analysis.”" is an oxymoron. Executable checking is the opposite of static code analysis. The statement "examined source code to look for patterns" is static code analysis not unit testing. This article calls it simulation but simulation is not quite unit testing. I can't tell from that article if the inputs were provided by a unit test or by a human. – Moby Disk May 8 at 15:48
12

Prior to the creation of C, was there any unit-testing? Did any assemblers include a testing a framework?

Erm. There is no specific relation between C and testing. C as a language doesn't feature any mechanics for (unit) testing. Testing was and is a distinct item in its own right. Like for many other components of development, more generic tools and procedures have been developed and provided. Nowadays often seen as natural parts of a language, as the are presented in an integrated way - hence the common name Integrated Development Environment - but they are only made to look that way, they are still just sold as a package.

Of course, there are languages that naturally ease some aspect of automated testing - usually as side effect due the formal way of specification they employ to reduce undesired effects. The same information can be used seamless for certain parts of verification and test automation. I'm for one a great fan of Ada (*1). It's way of clear defined data and operations thereon allows not only detection of unsafe or faulty constructs during compilation, but as well on demand addition of test.

Another great example is VHDL, as here the definition of a test bench is (almost) mandatory and part of the language. So while a test framework is part of VHDL, it also shows that any framework (beyond simple, formal checking), goes beyond what a language can do and is always application based.

I can find some academic journals on this topic -- most of which I don't have access too, but I don't see any implementations or examples of it.

Because testing always has been and forever will be a domain of application specific tests. While some languages are (or have developed to be) supportive of test scenarios, it's the application environment that defines the how and wherewith. This is the same way true for unit tests as complete runs.

Thus tests are not only language independent, but also technology independent. Testing goes goes back into the age of mechanical punch card operation. I remember porting an application from stand alone (batch) /370 Assembly, operating on punch cards to a disk based environment (as well /370 Assembly - but we finally got red of all the highly opaque manual/scripted batch). These were the early 1980s. One fun part was that this batch application itself has been converted some 10+ years before from mechanical punch card to /360 Assembly.

Among the documentation I found a set of card stacks, various plug sheets a one special page: the test bench for the run. It was a rather complex setup with several plug sheets involved and multiple tabulator runs. Whenever it was set up for production these stacks were feed to the tabulators to produce an output stack. This output stack, and one of the test stacks was then to be fed to a collator plugged (the collator plug schematic was part of the documentation) to compare both stacks. The documentation stated that this had to be done before each and every run were the machines have been used otherwise inbetween.

Long story short: Test benches have always been a thing and there has been automated support even in the dark ages of mechanical computing. The difference is like with all our too in vastly grown resources that allow more support than ever before.


*1 - Yes, Ada and Assembler do get well together in my mind, as both allow almost unlimited freedom in design. One just needs to specify what is intended. In the end they only differ that in Assembly one can talk dirty without coming clean, while in Ada everything has to be spelled out in a calm and polite manner :))

| improve this answer | |
  • 1
    Your answer sounds like it is about "testing" which is a human interacting with the software to test it, while the question is specifically about "automated unit testing" which is about writing software that automatically tests the code without human interaction. UPDATE: Looks like the question was updated to be clear, so there are several answers that didn't assume "unit testing" == "automated unit testing" – Moby Disk May 8 at 15:51
  • @MobyDisk Well, the question did change wording over time, didn't it? – Raffzahn May 8 at 15:54
  • 1
    Worth mentioning that Ada is a pioneer of feature testing; backed by the extensive ACATS test suite to guarantee any Ada compiler conforms to the language specification. Not quite unit testing as the whole compiler is under test; but each test is specific to one feature. Not 1972 but still pretty revolutionary in 1980. – Brian Drummond May 8 at 19:52
12

Was there any Unit-Testing prior to 1972?

Yes there was. Only a fool would put a system together out of untested and undebugged parts and then try to make the system work. No framework is needed: you write some code, you test it in isolation: that's unit testing by original definition. Now you can combine your units into the next level of thing.

In the mid-1970s (ok, not before 1972, but the art doesn't change that quickly) I wrote a PDP-11 driver for a synchronous communication device, which I debugged by wrapping it in some code that simulated at some level the operation of the hardware. That's a unit test. The driver basically worked by the time it saw real hardware. The test structure got thrown away (well, packed into my bag of useful pieces) once the driver worked; from that point it was tested in use.

You may be confusing "automated unit testing" with "unit testing".

| improve this answer | |
  • 3
    I see the question has been changed to explicitly be about automated unit testing. Nevertheless, I'll leave this answer here. – another-dave May 8 at 12:34
  • Is that unit testing, or is it mocking? Mocking is obviously used extensively in unit testing, but if you're testing the full program against a simulated or "mocked" hardware I think that's subtly different. I have worked on power system controls where we did that, but we always tested the entire controls logic on the simulated system rather than a testing each function of the control logic. – David Waterworth May 9 at 1:36
  • 1
    I don't recall at this remove, but the driver is one file, thus "a unit", though I surely proceeded routine-wise. And from my point of view then, the driver was only part of a program (an emulation of a Univac 9000 remote batch station). The terms are IMO not precisely defined. – another-dave May 9 at 2:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.