16

I was curious if there was a benchmark test that was available to compare CPUs that are 8-bit. People always like meaningless analysis to justify some decision, and I was curious if there was a historically relevant example. I found in literature the compression of Shakespeare using LZ, but I wasn't sure if there was a "main stream" comparison from the 8-bit era of computing.

2
  • 7
    Clock frequency did the job pretty well, back then. – fraxinus Feb 16 at 8:47
  • I remember unprofessionally trying Mandelbrot images. (You could come back from school and something had been attempted by the practically steampunk CPU) – David Tonhofer Feb 17 at 17:04
24

There are three primary benchmarks used during this period. They are not strictly "CPU" tests, but were often used for that purpose.

  1. The almost unknown Rugg/Feldman suite from 1977. This was a series of seven, and later eight, small BASIC benchmarks. They were clever in that each test was a modification of the previous so you only had to type in one or two lines - important in the terminal era. Although almost unknown in the US by 1980, it went on to have a much longer life in the UK where it became a standard of sorts.

  2. The Byte Sieve, likely the most widespread. Intended to be a language benchmark, it was almost always seen as a machine benchmark using assembler versions of the code. It was seen in many languages on lots of platforms. Unfortunately, collecting data is somewhat difficult because the authors sorted in... an odd way. It was still in use well into the 1980s and likely the one you want to use if you're running a test today.

  3. The Creative Computing test, another all-BASIC one but generally (and incorrectly!) often used as a machine benchmark.

8

For assessing floating-point performance, there was the Savage benchmark. It was proposed by Bill Savage of Microfloat in Houston, Texas, and published in the Ray Duncan column "16-bit Software Toolbox" in Dr. Dobb’s Journal, Number 83, September 1983, p. 120. A scan of the entire Dr. Dobb's volume can be retrieved here; the relevant article starts at page 561 of the PDF file.

The benchmark assigns a=1 and then times 2500 repetitions of a=tan(atan(exp(ln(sqrt(a*a)))))+1. With the software implementations of floating-point arithmetic available on 8-bit machines this typically took several minutes to complete, if memory serves.

2
  • This was actually more a benchmark of the transcendental math library (optimization quality), and not just (or maybe even mostly) processor or floating point performance. IIRC, at least one microcomputer Basic implementation had a really slow math library, compared to Microsoft's, on the same 8-bit CPU. – hotpaw2 Mar 15 at 18:45
  • From personal experience, whether justified or not, this was a popular way to assess the floating-point performance of microcomputers. Since none of these had hardware support for floating-point arithmetic, it was necessarily a test of a mixture of software efficiency and CPU performance. Just like the later popular Dhrystone was mostly a test of the efficiency of string processing primitives. – njuffa Mar 15 at 18:57
5

The circa-1981 Byte Sieve benchmark may have originally been conceived as a programming language benchmark. But the algorithm consisted of such a simple set of intrinsic basic operations (adds, compares, indexed or calculated address load/stores, not much more) that it could easily be hand-coded or hand-translated to 8-bit ISA assembly language, and thus used to compare 8-bit processor system performance doing such basic ops.

(* but with current 32/64-bit multi-issue and multi-level-cached processors, hand-coding may or may not be as comparable across ISAs and implementations.)

3
  • 1
    "not be as comparable"? Do you mean "not be as representative of typical software"? That's probably true (the access patterns for a sieve are very sensitive to cache sizes, and other workloads might depend more on ALU and branch prediction), but I see no reason why you couldn't still compare the speed at which you could get a Sieve of Eratosthenes to run on a given system, with hand-tuned asm. e.g. maybe using SIMD to set multiple bits at once in the sieve when iterating to mark off small primes. Or did you mean with tuning being harder, level of tuning skill/effort can skew results more? – Peter Cordes Feb 16 at 8:36
  • @Peter I'd imagine that with today's machines you'd only test memory throughput with such a benchmark and not the CPU. – Voo Feb 17 at 12:24
  • @Voo: yeah for large sieves that don't fit in cache that's probably true, testing memory-level parallelism for strided RMW. But L1d and maybe L2 cache can be fast enough to keep up with the amount of work it takes to do bit-indexing into a byte or word for a packed bitmap. (Using one bool per byte is faster for very small sieves where even that 8x wasteful format still fits in cache, and yeah that would just test memory.) – Peter Cordes Feb 17 at 12:27
3

There were several, but the Sieve benchmark is probably one of the most famous.

6
  • 3
    Was it used to compare CPUs? Initially it was a language comparison benchmark AFAICT. – Stephen Kitt Feb 15 at 15:24
  • 2
    Can you expand? – user3840170 Feb 15 at 15:42
  • 2
    @StephenKitt - and, of course, nobody ever misused a benchmark now, did they? – Jon Custer Feb 15 at 22:25
  • @JonCuster of course benchmarks get misused, I’m wondering if there were any published 8-bit CPU comparisons which relied on sieve (as opposed to publications of other comparisons, including assembler/compiler/interpreter comparisons, which could perhaps be used to compare CPUs). – Stephen Kitt Feb 15 at 22:43
  • @StephenKitt - sorry, I forgot the smiley face there - tongue was firmly in cheek. Nobody in my lab believed any of the benchmarks of the time, except us running our code on a machine. But PCs were mainly to collect data, not crunch numbers. – Jon Custer Feb 16 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.