Why does _Beneath Apple DOS_ show FM encoding rather than GCR?

Question

From the beginning in 1978 the Apple II floppy disk system used GCR encoding to convert data bytes to sequences of binary values to be written to the diskette: first 4-and-4 encoding (never released), then 5-and-3 encoding (DOS 3.1 through 3.2.1 13-sector), and later 6-and-2 encoding (DOS 3.3 16-sector).

Starting on page 3-3¹ of Worth and Lechner's Beneath Apple DOS they describe the format of bits on the track. They mention that they're discussing the actual analogue signals, only the digital 1s and 0s, and say that, "For the purposes of this discussion the demarcation of these intervals will be depicted by a clock bit," indicating that what they describe might be slightly different from what's really being written to or read from the diskette.

However, what they describe there and show in figure 3.2 is clearly FM encoding, with a clock bit before each data bit, rather than GCR:

Is this just flat-out wrong, or oversimplified, or am I missing something here? If it's not simply wrong, how does this affect the later discussion of normal bytes vs. self-sync bytes and synchronization in the gaps between the fields, particularly with regard to figure 3.10 on page 3-9, showing how one can start reading at any bit in a series of (10-bit) sync bytes and recover the byte alignment within 5 bytes?

---

¹_{I've linked to the 1982 Fourth Printing edition here, but the content of this section is the same in substance in the 1983 Fifth Printing, "Updated for Apple IIe."}

Answer 1

While it is "flat out wrong", they do preface this with:

As this manual is primarily concerned with software, no attempt will be made to deal with the specifics of the hardware. For example, while in fact data is stored as a continuous stream of analog signals, we will deal with discrete digital data, i.e. a or a 1 . We recognize that the hardware converts analog data to digital data but how this is accomplished is beyond the scope of this manual.

The diagram and surrounding text were updated in the Fifth Printing, March 1983:

Data bits are recorded on the diskette in precise intervals. The hardware recognizes each of these intervals as either a or a 1 . We will define these intervals to be "bit cells". A bit cell can be thought of as the distance the diskette moves in four machine cycles, which is about four microseconds. Using this representation, data written to and read back from the diskette takes the form shown in Figure 3.2. The data pattern shown represents a binary value of 101.

The figure they refer to is actually now Figure 3.3:

As for why the authors originally showed FM encoding, we can only presume that they were aware that current disk drives used that and unaware that the Disk II didn't. It's probably not a coincidence that the above correction to the text was made around the same time that their publisher was about to release Sather's Understanding the Apple II, which contained the first complete and detailed analysis of the disk system hardware.

Answer 2

The first figure is flat-out wrong. Only the "D" bits they've marked are stored on the diskette as the presence (D=1) or absence (D=0) of a magnetic flux change. Those clock bits exist neither on the diskette nor in the internals of the controller. (The controller does have its own separate clock signals, but these are independent and never intermixed with the data bits.)

The second digram gives an accurate picture of what's stored on the diskette: a stream of 1 and 0 bits encoded as a flux change or lack of a flux change.

That said, as well as GCR encoding for the sector data fields, the Disk II system did use FM encoding for the sector address fields as is shown later in Beneath Apple DOS and below. To understand how this worked we need to understand how the controller (with some assistance from software) reads data from the diskette and divides it up into bytes.

Byte Framing

To start reading, the shift register that holds the read data is cleared (set to $00) and the controller starts in a state that Jim Sather calls "QA WAIT,"¹ where it is waiting for a 1 bit (i.e., a flux transition) to appear. Once a 1 appears, it clocks that and next seven bits (1s or 0s) into the shift register. The software is expected to continuously test bit 7 of the shift register for the appearance of that initial 1 bit and take that as the signal that a full byte has been read. In the mean time, after it finishes reading a byte the controller has re-entered QA WAIT and will leave the shift register alone until another 1 bit appears. Shortly after that it will clear the shift register and start clocking in another byte.

This explains why bytes stored on the disk must always start with a 1 bit: that's what triggers the start of the shift register load sequence. When starting to read a series of bytes the controller will start at a random bit (most likely in the middle of a byte); the "autosync" sequence of bits in the second figure in the question shows how, via ignoring 0 bits when starting a byte read, the controller can sync to the byte boundaries of data being read from the diskette. (The 0 values in that figure not covered by the braces underneath are the ones ignored while in the QA WAIT state.)

That bytes must start with a 1 bit is particular to how the Disk II controller assembles bytes. The other major limitation, that there may not be more than two 0 bits in a row within a byte, is a general limitation of reading self-clocked data from a variable speed medium.²

FM vs. GCR Encoding

The two major limitations above determine that only a subset of all bytes valued $00 to $FF can be reliably read from a diskette; the byte values that can be used always start with a 1 in the most significant bit and never have more than two (or in DOS 3.2 or earlier, one) 0 bits in a row.

This allows for use of both FM and GCR encoding, and in fact Apple DOS uses both. On page 3-12 of Beneath Apple DOS they show the format of the address field for each sector:

As you can see, the bytes encoding the volume, track, sector and checksum are in a version of FM encoding: each byte is a series of 1 bits alternating with data bits. (The data bits are ordered the way they are to make decoding easier: left-shift the XX byte, loading 1 into the LSB, and AND the result with it with the YY byte.)

For the data field for each sector, however, a GCR encoding is used.

Note that the sync, prologue, and epilogue bytes are not "encoded" at all; they are simply raw bit patterns that follow the rules above but are invalid in either encoding,³ allowing the system to recognize them as "control bytes" rather than any kind of data.

Other Encodings

From the above you can see that the Disk II system seems also, with appropriate software, probably capable of reading diskettes using Shugart's standard single-density soft-sectored formats that use FM encoding, such as 18 × 128 byte sectors per track (78.75 KB).⁴

However, the limitations above also indicate why an Apple II cannot read any of the double-density MFM standard formats: they encoded data in such a way that some bytes could start with a 0, and the Disk II controller cannot sync to those (though a different controller could be designed to do so using the same drive).

---

¹_{Jim Sather, Understanding the Apple II, Figure 9.16. Chapter 9 is a good but very detailed (42 dense pages) description of the Disk II hardware and software; here I attempt to give an accurate but limited description of just the parts relevant to the question.}

²_{The limitation for DOS 3.2 and earlier is one 0 bit; this is due to using a different state machine in the first generation of controllers. Generically, the particular number of 0 bits in a row that are allowed will depend on the design of a particular system and how much accuracy it loses as it goes longer without flux transitions, but system designers will always set a particular limit.}

³_{Actually the middle $AA is valid FM of half a byte, but would never be read as such because the preceding byte is not valid FM. The trailing $EB is actually never fully written; see Understanding the Apple II for details.}

⁴_{The data rate must also be the same, of course. This is limited by the media itself, which is probably why common controllers such as the Western Digital FD1791 and Fujitsu MB8866 use the FM data rate established by the IBM 3740 format: 125 kHz, or 4 μs for the clock bit and 4 μs for the data bit. The Disk II system uses 4 clock cycles per bit, which at 1.023 MHz is very close to 4 μs.}