Which is the first version of DOS to support more than one reserved sector in a FAT file system?

Question

I'm trying to fix a problem when trying to use DOS stuff on SSDs. The problem is sector alignment in an SSD is completely different from what DOS expects, and writes to FAT do funny things when the computer crashes in the middle that just don't match what FAT repair utilities can repair.

So I laid out a FAT filesystem with hand-computed offsets so the FAT start and root directory start and each cluster start would correspond to the correct sector boundary. (Right now I laid it out at 1K although new SSDs will be bigger than that.) Then I tried it. It failed rather catastrophically. MS-DOS 2.0 and Windows 95 disagree on the contents of the disk. As in, I copied MORE.COM to the disk using MS-DOS 2.0 and tried to read it back in Windows 95 boot floppy and it wasn't there. I tried to then write a file using Windows 95 to see where it went. Sure enough it went somewhere else.

Hilariously, my Linux host happens to agree with Win95; however it exhibits a bug where it doesn't stop directory entry scanning on encountering a 0 as the first character.

Hexdump and analysis:

00000c00  eb 3c 90 6d 6b 64 6f 73  66 73 20 00 02 08 02 00  |ë<.mkdosfs .....|
    No it's not really mkdosfs
    Size of sector = 512 bytes (most DOS versions don't like logical sectored fat.)
    Sectors per cluster = 8
    Reserved logical sectors = 2
00000c10  02 00 02 9a 7f f8 0c 00  3f 00 10 00 06 00 00 00  |.....ø..?.......|
    Number of FATs = 2
    Root dir entries = 512
    Media Descriptor = 0xF8
    Logical Sectors per FAT = 12
00000c20  9a 7f 00 00 80 00 29 00  00 00 00 4e 4f 20 4e 41  |......)....NO NA|
00000c30  4d 45 20 20 20 20 46 41  54 20 20 20 20 20 be be  |ME    FAT     ¾¾|
    snip oops you can't boot this yet message
00000df0  00 00 00 00 00 00 00 00  00 00 00 00 00 80 55 aa  |..............Uª|
00000e00  00 f0 ff 00 00 00 00 00  00 00 00 00 00 00 00 00  |.ðÿ.............|
    DOS 2.0 thinks this is the first FAT sector
00000e10  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000  f0 ff ff ff 0f 00 00 00  00 00 00 00 00 00 00 00  |ðÿÿÿ............|
    Windows 95 thinks this is the first FAT sector; as do I when I formatted it
00001010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002600  00 f0 ff 00 00 00 00 00  00 00 00 00 00 00 00 00  |.ðÿ.............|
    DOS 2.0 thinks this is the first FAT sector of the second FAT
00002610  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002800  f0 ff ff ff 0f 00 00 00  00 00 00 00 00 00 00 00  |ðÿÿÿ............|
    Windows 95 thinks this is the first FAT sector; as do I when I formatted it
00002810  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00003e00  4d 4f 52 45 20 20 20 20  43 4f 4d 20 00 00 00 00  |MORE    COM ....|
00003e10  00 00 00 00 00 00 00 60  68 06 01 00 80 01 00 00  |.......`h.......|
00003e20  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    DOS 2.0 wrote its root dir entry here
*
00004000  44 55 4d 4d 59 20 20 20  54 58 54 20 00 00 00 00  |DUMMY   TXT ....|
00004010  00 00 55 56 00 00 71 9f  55 56 02 00 36 00 00 00  |..UV..q.UV..6...|
00004020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    Win95 wrote its root dir entry here
*
00006e00  b4 30 cd 21 86 e0 3d 00  02 73 09 ba fc 01 b4 09  |´0Í!.à=..s.ºü.´.|
    snip rest of MORE.COM
*
00008000  48 69 20 74 68 65 72 65  2e 20 49 27 6d 20 73 6f  |Hi there. I'm so|
00008010  6d 65 20 74 65 78 74 20  63 72 65 61 74 65 64 20  |me text created |
00008020  74 6f 20 74 65 73 74 20  64 69 73 6b 20 61 63 63  |to test disk acc|
00008030  65 73 73 2e 0d 0a 00 00  00 00 00 00 00 00 00 00  |ess.............|
00008040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    Win95 wrote the file I created here. This is where I think the first cluster is.
*

I only have a couple versions of DOS right now and can't probe. At one point I had intended to gather up as many as I could to replace the mostly failing bootdisk.com but that's a long ways off.

If somebody really wants to attack this by probing rather than the quick way of knowing the answer, here's a 20mb disk image with an empty filesystem. It's been bzip2 compressed and base64 encoded. I pretty much had to use bzip2 here because other compressors I tried don't compress long strings of zeros good at all.

begin-base64 644 hda-Joshua.img.bz2
QlpoOTFBWSZTWbYfIa0CeNF////PaYBgMSAU/wfeID8rnibsGUARQhxULdEC
QEBQGXl9sAFYgQlVPFTYhqaaeiNNNqDQGgADQGgMmQYGoaaYg0xPRBxoaGho
BoDEDQGQAAaaABoBkAAADCSKapoHqANAAAAAADQAA0A0AAAHqVZS1X5hsako
kipGTsAprISdkhJJJCSBneAKAwIBAJJBPDLzhHEISQAkkFO9Q1GQeN9hi3Gc
aWShwndLorrWMinDTWEJIASTwHO1L8Q8hIBZEzbXvTpvy2wlA0fxXcsVoTYw
6WEtQxJ8HyJQ8/MioRgeSyaVO7xmjHL2FSB0VX6F4RESJkzGhRcnk4xiGTxK
xXnihRJYFBSoxUKXFb8B9TFqCVS3CkQRK8cq+eVhSpL+I3Ucw9EzjyRAiPZP
mzzDRDfEViuhXM9SzHI2o+VxtkqwWUcGEUfujE+px6Vab5+n8wUOKuuLtMX7
Ip6OxWmMBZui5We4/1+sisFrFouQL1v1umULLCxadgyC27IGsTHgs3GTdFXn
U3L5J+kECSAEkgl1/ofx7pSDMQ/xdyRThQkLYfIa0A==
====

Answer 1

Since I just so happen to have a battery of floppy images with various DOS kernels I compiled to investigate a somewhat similar problem, I decided to try the probing route. I did not use the image in the question, though; instead, I created the disk image attached below (also bzip2-compressed), geometry 32 × 16 × 32, and used that for testing.

begin-base64 644 hda.img.bz2
QlpoOTFBWSZTWRIY2P8BAU5/ffRWwADAOXAQsyuPAAIAUABAAgiQACIAEEAI
AEmwANmzDTSob1QyA0DQPSBmo0G9SDRCGjQnqYCZA0GAmICKSg0AAANAaAAH
yrDGOQn25IB7wIpbIUFlDYhUAVVZqyFEExCIAgAFeBswiqgAAxC51JMnTbwu
SSjJpggAAQAAAfUgmGUD7yE8VzvTcJmTETsUKNZTgRCrte8NEsYNS19xXVXN
hksyRjK8s+n1JqeeCfSbc7mxwo+WhR+Id1y+VQIiIgAAAPxQHI/i7kinChIC
Qxsf4A==
====

The partition was formatted with mkfs.fat 4.2 and then manually edited in a hex editor to use a more compatible OEM ID and scrub the boot error message. I mounted the above image as a virtual disk in QEMU, and the test kernel image in the floppy drive. I booted the kernel from the floppy and then ran type c:test at the command prompt. If the command succeeded, I presumed the kernel reads the drive correctly.

All the DOS kernels I tested detected the partition, but the earliest version of MS-DOS that succeeded in reading the file was MS-DOS 5.0; MS-DOS 2.0 through 4.0 could not find it. PC DOS was similar, unsurprisingly, as the two were still different builds of the same kernel back then: 5.0 worked, 4.01 did not. The file was also read successfully in DR DOS as early as version 3.40. Also unsurprisingly, the file reads correctly in FreeDOS.

With the versions that failed to read the file, I also tried writing to the disk image, which revealed an interesting divergence: PC DOS 2.0 not only failed to account for more than one reserved sector declared in the BPB, it was apparently also convinced that the FAT has only 6 sectors instead of 12 as the BPB declared. It also had no problems allocating cluster #1 to the newly-created file, even though cluster numbering starts from 2. (That’s right: cluster #1 overlaps the root directory area.) The later MS-DOS 3.30 through 4.0 only have the former flaw.

Answer 2

This smelled like a problem with the two operating systems disagreeing on where the filesystem actually started because they are exactly a sector different in where they think the metadata is, and I suspected that the difference was between the CHS and LBA information in the partition table. So let's look at it:

$ fdisk disk.img
Disk: disk.img  geometry: 160/4/63 [40320 sectors]
Signature: 0xAA55
         Starting       Ending
 #: id  cyl  hd sec -  cyl  hd sec [     start -       size]
------------------------------------------------------------------------
*1: 01    0   0   7 - 1023 255  63 [         6 -      32666] DOS FAT-12
 2: 00    0   0   0 -    0   0   0 [         0 -          0] unused
 3: 00    0   0   0 -    0   0   0 [         0 -          0] unused
 4: 00    0   0   0 -    0   0   0 [         0 -          0] unused

The partition start passes muster: CHS (0,0,7) is LBA 6. The end looks very wrong because CHS (1023,255,63) is bigger than the geometry, but that's not actually the problem here.

Per e.g. §6.2 of Large Disk HOWTO, "A well-known claim says that partitions should start and end at cylinder boundaries." and "MSDOS 6.22 has an alignment requirement. Extended partition sectors that are not on a cylinder boundary are ignored by its FDISK. The system itself is happy with any alignment, but interprets relative starting addresses as if relative to an aligned address: The starting address of a logical partition is given relative not to the address of the extended partition sector that describes it, but relative to the start of the cylinder that contains that sector".

The detail is not quite as I remember and not entirely clear either, but the "partitions should start and end at cylinder boundaries" certainly is. Your partition table says it starts on sector 7, but DOS is (sometimes?) not paying attention to that field.

Meanwhile, modern hard disks use 4 KiB sectors, and although SSDs don't actually need to use a specific sector size because they use compression and other trickery so the logical sectors do not align with their internal storage anyway, they are intended as drop-in replacements for hard disks and pretend to be 512e for maximum compatibility. So you should certainly design your partition layout to ensure 4kiB alignment (rather than the 1kiB in your question), which means a multiple of 8 sectors.

To satisfy both DOS and modern disks' requirements, you need to find a partition start (c, h, 1) which maps to a multiple-of-8 LBA. Because there are usually 63 sectors per track, you are looking for a (c * head-count + h) which is divisible by 8. Thus the first suitable CHS is (0,8,1) which corresponds to an LBA of 504.