Where did the free parameters of IEEE 754 come from?

Question

It's clear where many of the design decisions of IEEE 754 floating point come from. For example, the binary format maximizes efficiency on binary hardware. And 32 and 64 bits for single and double precision make sense on machines with power of two word sizes.

But the number of bits for sign+exponent is a free parameter, and the actual values chosen, being 9 and 12 respectively, seem terribly random. To the extent I would've any prior expectation, I would've expected them to be powers of two, like 8 or 16, to make it easier to implement them in software, since the standard was written at a time when most computers did not have floating-point hardware.

Where did those values come from?

Answer 1

From an interview with Dr. William Kahan, the IEEE-754 formats were based on the VAX F and G formats, which have 8 and 11-bit exponents respectively. In fact Dr. Kahan also said that previously VAX has a double precision D format which has the same 8-bit exponent as the single precision F format which proved too limited in practical use, therefore DEC had to introduced 3 more bits to create the new G format to avoid the underflow/overflow issues

But why did DEC use 8 bits for the exponent in the F and D formats? The reason is to be able to represent all important physical constants, including the Plank constant (6.626070040 × 10^-34) and the Avogradro constant (6.022140857 × 10²³), as stated in PDP-11/40 Technical Memorandum #16

Another rationale for the 64-bit format as described by David Stephenson ("A Proposed Standard for Binary Floating-Point Arithmetic", IEEE Computer, Vol. 14, No. 3, March 1981, pp. 51-62) is that

For the 64-bit format, the main consideration was range; as a minimum, the desire was that the product of any two 32-bit numbers should not overflow the 64-bit format. The final choice of exponent range provides that a product of eight 32-bit terms cannot overflow the 64-bit format — a possible boon to users of optimizing compilers which reorder the sequence of arithmetic operations from that specified by the careful programmer.

In fact nowadays the rule for IEEE-754 interchange format the size for the exponent is round(4 log₂(k)) − 13 bits so every time we double the width of the type, the exponent will be have ~4 more bits which allows for 16 multiplications of the narrower type without overflow

The encoding scheme for these binary interchange formats is the same as that of IEEE 754-1985: a sign bit, followed by w exponent bits that describe the exponent offset by a bias, and p − 1 bits that describe the significand. The width of the exponent field for a k-bit format is computed as w = round(4 log₂(k)) − 13. The existing 64- and 128-bit formats follow this rule, but the 16- and 32-bit formats have more exponent bits (5 and 8) than this formula would provide (3 and 7 respectively).

https://en.wikipedia.org/wiki/IEEE_754#Interchange_formats

Those are the things I summarized from the below links which you should read to get more details

• Relationships between 128, 64, and 32 bit IEEE-754 floating point numbers
• Why do higher-precision floating point formats have so many exponent bits?
• What is the rationale for exponent and mantissa sizes in IEEE floating point standards?
• Why did IEEE754 choose 11 exponent bits for double aka binary64?
• How are IEEE-754 single and double precision formats determined?

Answer 2

Well, the first thing to remember about these binary formats you're talking about (there were also decimal formats) is that they are the interchange formats; it's not required that hardware, or even software, use these for internal calculations. It's perfectly reasonable for an implementation to use an entire separate byte for the sign and one or more separate bytes for the exponent, if that's felt to be a good memory vs. speed tradeoff by the designer. (And in fact that's exactly what many 6502 floating point routines did!)

The binary interchange formats were chosen first to fit into common modern binary machine words: 16, 32, 64, 128 and 256 bits. The next decision to be made is the balance between exponent bits and significant bits: too many of the former and you have a format with perhaps more range you need and not enough precision; too many of the latter and you have the opposite problem.

An exponent of eight bits as used by binary32 (single-precision) interchange format gives you a binary exponent range of only −126 to +127, which limits the absolute sizes of your values to somewhere around 10³⁸ decimal. That's not a bad range, but it's certainly not enough for many applications. Reducing it to 7 bits (so that the sign and all exponent bits could be in a single byte) would make the problem even worse, and probably wouldn't help much with calculations in software since you still have to split out the sign, and anybody concerned about speed is going to use an internal format with a separate byte for that anyway.

Once we move to binary64 (double-precision), the problem goes in the opposite direction: increasing the 11-bit exponent to a 16-bit exponent would give an enormous absolute range of something like 10⁵⁰⁰⁰ decimal and, because it's "stealing" 5 bits that could otherwise be used for more precision, would reduce precision from almost 16 digits to just over 14. The current scheme was felt to be a better tradeoff.

Following is a table I made a while back when I was thinking about this topic myself. p is precision (in bits) of significand, including an implicit leading 1. prec is number of digits of decimal precision (c*log₁₀(2)), demax is the maximum exponent in decimal (2^(e-1)-1 * log₁₀(2)).

byt bits p   e   prec    demax      notes
2   16  11   5   3.31     4.51      IEEE 754 half precision (not basic)
3   24  16   8   4.81    38.23
3   24  17   7   5.11    18.96
3   24  18   6   5.42     9.33
3   24  19   5   5.72     4.51
4   32  24   8   7.22    38.23      IEEE 754 single precision
5   40  32   8   9.63    38.23
5   40  31   9   9.33    76.76
5   40  30  10   9.03   153.8
6   48  40   8  12.04    38.23
6   48  39   9  11.74    76.76
6   48  38  10  11.44   153.8
7   56  48   8  14.45    38.23
8   64  56   8  16.86    38.23
8   64  53  11  15.95   308.0       IEEE 754 double precision

MS 6502 BASIC shipped with 32-bit 6 digit FP (8 KB) in 1976 (8 KB), later expanded to 40-bit 9 digit (9 KB) in 1977.

Answer 3

I can see logic in selecting 23 mantissa bits for 32-bit float: together with the implied "one" it fits exactly in 24 bits, which is convenient for emulating floats on a purely integer processor.

And another pressure is the precision -- they had to trade precision for exponent size, and the precision (which was not particularly high for float-32) won a bit or two.

64-bit float decisions might be more random since the precision it gives and the maximum/minimum exponent it allows seemed to be less of importance at the time (both were perceived to be "more than enough", and the exponent still is).

Handling both exponent sizes within a 16-bit cpu (which was i8086 for intel) was obviously easy.