15

I was curious if there was a benchmark test that was available to compare CPUs that are 8-bit. People always like meaningless analysis to justify some decision, and I was curious if there was a historically relevant example. I found in literature the compression of Shakespeare using LZ, but I wasn't sure if there was a "main stream" comparison from the 8-bit era of computing.

Improve this question
2
  • 7
    Clock frequency did the job pretty well, back then.
    – fraxinus
    Feb 16, 2021 at 8:47
  • I remember unprofessionally trying Mandelbrot images. (You could come back from school and something had been attempted by the practically steampunk CPU)
    – David Tonhofer
    Feb 17, 2021 at 17:04

4 Answers 4

Reset to default
24

There are three primary benchmarks used during this period. They are not strictly "CPU" tests, but were often used for that purpose.

  1. The almost unknown Rugg/Feldman suite from 1977. This was a series of seven, and later eight, small BASIC benchmarks. They were clever in that each test was a modification of the previous so you only had to type in one or two lines - important in the terminal era. Although almost unknown in the US by 1980, it went on to have a much longer life in the UK where it became a standard of sorts.

  2. The Byte Sieve, likely the most widespread. Intended to be a language benchmark, it was almost always seen as a machine benchmark using assembler versions of the code. It was seen in many languages on lots of platforms. Unfortunately, collecting data is somewhat difficult because the authors sorted in... an odd way. It was still in use well into the 1980s and likely the one you want to use if you're running a test today.

  3. The Creative Computing test, another all-BASIC one but generally (and incorrectly!) often used as a machine benchmark.

Improve this answer
8

For assessing floating-point performance, there was the Savage benchmark. It was proposed by Bill Savage of Microfloat in Houston, Texas, and published in the Ray Duncan column "16-bit Software Toolbox" in Dr. Dobb’s Journal, Number 83, September 1983, p. 120. A scan of the entire Dr. Dobb's volume can be retrieved here; the relevant article starts at page 561 of the PDF file.

The benchmark assigns a=1 and then times 2500 repetitions of a=tan(atan(exp(ln(sqrt(a*a)))))+1. With the software implementations of floating-point arithmetic available on 8-bit machines this typically took several minutes to complete, if memory serves.

Improve this answer
4
  • This was actually more a benchmark of the transcendental math library (optimization quality), and not just (or maybe even mostly) processor or floating point performance. IIRC, at least one microcomputer Basic implementation had a really slow math library, compared to Microsoft's, on the same 8-bit CPU.
    – hotpaw2
    Mar 15, 2021 at 18:45
  • From personal experience, whether justified or not, this was a popular way to assess the floating-point performance of microcomputers. Since none of these had hardware support for floating-point arithmetic, it was necessarily a test of a mixture of software efficiency and CPU performance. Just like the later popular Dhrystone was mostly a test of the efficiency of string processing primitives.
    – njuffa
    Mar 15, 2021 at 18:57
  • I just ran this in Python3 on a MacBook Pro with an i9. It finished so fast I didn't think it worked. It averages 1.5 msec running it a few times. Program: from math import tan, atan, exp, log, sqrt; a=1 \n for x in range(2500): a=tan(atan(exp(log(sqrt(a*a)))))+1 \n print(a)
    – Ed Griebel
    Sep 25, 2021 at 5:23
  • 1
    @Ed Griebel This isn't very surprising. Consider the operating frequency of a 4 MHz Z80 vs a modern 4 GHz x86-64 processor. That's a factor of 1000x. Now consider that a basic floating-point operation (e.g. multiply) emulated in software took a few thousand cycles on the Z80 but only a few cycles in modern hardware. This gives us another factor of 1000x. So a factor of 1,000,000 altogether. Obviously this is only a rough back-of-the-envelope number, but it provides a reasonable ballpark figure.
    – njuffa
    Sep 25, 2021 at 6:02
6

The circa-1981 Byte Sieve benchmark may have originally been conceived as a programming language benchmark. But the algorithm consisted of such a simple set of intrinsic basic operations (adds, compares, indexed or calculated address load/stores, not much more) that it could easily be hand-coded or hand-translated to 8-bit ISA assembly language, and thus used to compare 8-bit processor system performance doing such basic ops.

(* but with current 32/64-bit multi-issue and multi-level-cached processors, hand-coding may or may not be as comparable across ISAs and implementations.)

Improve this answer
3
  • 1
    "not be as comparable"? Do you mean "not be as representative of typical software"? That's probably true (the access patterns for a sieve are very sensitive to cache sizes, and other workloads might depend more on ALU and branch prediction), but I see no reason why you couldn't still compare the speed at which you could get a Sieve of Eratosthenes to run on a given system, with hand-tuned asm. e.g. maybe using SIMD to set multiple bits at once in the sieve when iterating to mark off small primes. Or did you mean with tuning being harder, level of tuning skill/effort can skew results more?
    – Peter Cordes
    Feb 16, 2021 at 8:36
  • @Peter I'd imagine that with today's machines you'd only test memory throughput with such a benchmark and not the CPU.
    – Voo
    Feb 17, 2021 at 12:24
  • @Voo: yeah for large sieves that don't fit in cache that's probably true, testing memory-level parallelism for strided RMW. But L1d and maybe L2 cache can be fast enough to keep up with the amount of work it takes to do bit-indexing into a byte or word for a packed bitmap. (Using one bool per byte is faster for very small sieves where even that 8x wasteful format still fits in cache, and yeah that would just test memory.)
    – Peter Cordes
    Feb 17, 2021 at 12:27
3

There were several, but the Sieve benchmark is probably one of the most famous.

Improve this answer
6
  • 3
    Was it used to compare CPUs? Initially it was a language comparison benchmark AFAICT.
    – Stephen Kitt
    Feb 15, 2021 at 15:24
  • 2
    Can you expand?
    – user3840170
    Feb 15, 2021 at 15:42
  • 2
    @StephenKitt - and, of course, nobody ever misused a benchmark now, did they?
    – Jon Custer
    Feb 15, 2021 at 22:25
  • @JonCuster of course benchmarks get misused, I’m wondering if there were any published 8-bit CPU comparisons which relied on sieve (as opposed to publications of other comparisons, including assembler/compiler/interpreter comparisons, which could perhaps be used to compare CPUs).
    – Stephen Kitt
    Feb 15, 2021 at 22:43
  • @StephenKitt - sorry, I forgot the smiley face there - tongue was firmly in cheek. Nobody in my lab believed any of the benchmarks of the time, except us running our code on a machine. But PCs were mainly to collect data, not crunch numbers.
    – Jon Custer
    Feb 16, 2021 at 14:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .