20

I'm trying to fix a problem when trying to use DOS stuff on SSDs. The problem is sector alignment in an SSD is completely different from what DOS expects, and writes to FAT do funny things when the computer crashes in the middle that just don't match what FAT repair utilities can repair.

So I laid out a FAT filesystem with hand-computed offsets so the FAT start and root directory start and each cluster start would correspond to the correct sector boundary. (Right now I laid it out at 1K although new SSDs will be bigger than that.) Then I tried it. It failed rather catastrophically. MS-DOS 2.0 and Windows 95 disagree on the contents of the disk. As in, I copied MORE.COM to the disk using MS-DOS 2.0 and tried to read it back in Windows 95 boot floppy and it wasn't there. I tried to then write a file using Windows 95 to see where it went. Sure enough it went somewhere else.

Hilariously, my Linux host happens to agree with Win95; however it exhibits a bug where it doesn't stop directory entry scanning on encountering a 0 as the first character.

Hexdump and analysis:

00000c00  eb 3c 90 6d 6b 64 6f 73  66 73 20 00 02 08 02 00  |ë<.mkdosfs .....|
    No it's not really mkdosfs
    Size of sector = 512 bytes (most DOS versions don't like logical sectored fat.)
    Sectors per cluster = 8
    Reserved logical sectors = 2
00000c10  02 00 02 9a 7f f8 0c 00  3f 00 10 00 06 00 00 00  |.....ø..?.......|
    Number of FATs = 2
    Root dir entries = 512
    Media Descriptor = 0xF8
    Logical Sectors per FAT = 12
00000c20  9a 7f 00 00 80 00 29 00  00 00 00 4e 4f 20 4e 41  |......)....NO NA|
00000c30  4d 45 20 20 20 20 46 41  54 20 20 20 20 20 be be  |ME    FAT     ¾¾|
    snip oops you can't boot this yet message
00000df0  00 00 00 00 00 00 00 00  00 00 00 00 00 80 55 aa  |..............Uª|
00000e00  00 f0 ff 00 00 00 00 00  00 00 00 00 00 00 00 00  |.ðÿ.............|
    DOS 2.0 thinks this is the first FAT sector
00000e10  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000  f0 ff ff ff 0f 00 00 00  00 00 00 00 00 00 00 00  |ðÿÿÿ............|
    Windows 95 thinks this is the first FAT sector; as do I when I formatted it
00001010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002600  00 f0 ff 00 00 00 00 00  00 00 00 00 00 00 00 00  |.ðÿ.............|
    DOS 2.0 thinks this is the first FAT sector of the second FAT
00002610  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002800  f0 ff ff ff 0f 00 00 00  00 00 00 00 00 00 00 00  |ðÿÿÿ............|
    Windows 95 thinks this is the first FAT sector; as do I when I formatted it
00002810  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00003e00  4d 4f 52 45 20 20 20 20  43 4f 4d 20 00 00 00 00  |MORE    COM ....|
00003e10  00 00 00 00 00 00 00 60  68 06 01 00 80 01 00 00  |.......`h.......|
00003e20  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    DOS 2.0 wrote its root dir entry here
*
00004000  44 55 4d 4d 59 20 20 20  54 58 54 20 00 00 00 00  |DUMMY   TXT ....|
00004010  00 00 55 56 00 00 71 9f  55 56 02 00 36 00 00 00  |..UV..q.UV..6...|
00004020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    Win95 wrote its root dir entry here
*
00006e00  b4 30 cd 21 86 e0 3d 00  02 73 09 ba fc 01 b4 09  |´0Í!.à=..s.ºü.´.|
    snip rest of MORE.COM
*
00008000  48 69 20 74 68 65 72 65  2e 20 49 27 6d 20 73 6f  |Hi there. I'm so|
00008010  6d 65 20 74 65 78 74 20  63 72 65 61 74 65 64 20  |me text created |
00008020  74 6f 20 74 65 73 74 20  64 69 73 6b 20 61 63 63  |to test disk acc|
00008030  65 73 73 2e 0d 0a 00 00  00 00 00 00 00 00 00 00  |ess.............|
00008040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    Win95 wrote the file I created here. This is where I think the first cluster is.
*

I only have a couple versions of DOS right now and can't probe. At one point I had intended to gather up as many as I could to replace the mostly failing bootdisk.com but that's a long ways off.

If somebody really wants to attack this by probing rather than the quick way of knowing the answer, here's a 20mb disk image with an empty filesystem. It's been bzip2 compressed and base64 encoded. I pretty much had to use bzip2 here because other compressors I tried don't compress long strings of zeros good at all.

begin-base64 644 hda-Joshua.img.bz2
QlpoOTFBWSZTWbYfIa0CeNF////PaYBgMSAU/wfeID8rnibsGUARQhxULdEC
QEBQGXl9sAFYgQlVPFTYhqaaeiNNNqDQGgADQGgMmQYGoaaYg0xPRBxoaGho
BoDEDQGQAAaaABoBkAAADCSKapoHqANAAAAAADQAA0A0AAAHqVZS1X5hsako
kipGTsAprISdkhJJJCSBneAKAwIBAJJBPDLzhHEISQAkkFO9Q1GQeN9hi3Gc
aWShwndLorrWMinDTWEJIASTwHO1L8Q8hIBZEzbXvTpvy2wlA0fxXcsVoTYw
6WEtQxJ8HyJQ8/MioRgeSyaVO7xmjHL2FSB0VX6F4RESJkzGhRcnk4xiGTxK
xXnihRJYFBSoxUKXFb8B9TFqCVS3CkQRK8cq+eVhSpL+I3Ucw9EzjyRAiPZP
mzzDRDfEViuhXM9SzHI2o+VxtkqwWUcGEUfujE+px6Vab5+n8wUOKuuLtMX7
Ip6OxWmMBZui5We4/1+sisFrFouQL1v1umULLCxadgyC27IGsTHgs3GTdFXn
U3L5J+kECSAEkgl1/ofx7pSDMQ/xdyRThQkLYfIa0A==
====
10
  • 4
    Just trying to be sure, what is the sector alignment issue or what do you mean by it? What alignment DOS expects and what alignment the SSD has? As disks are just blocks of data that can be used in any way imaginable, how can therei even be an alignment issue? Can you provide the SSD geometry seen by DOS and seen by BIOS, to rule out other issues? E.g. is the disk partitioned and this is just a partition, or is this file system on disk without partitioning?
    – Justme
    Feb 22, 2023 at 6:00
  • 1
    Why does your hexdump start at 0xc00? Feb 22, 2023 at 9:00
  • 1
    @Justme: int13h always provides 512 byte sectors to hard disks but modern SSDs have physical sector alignment of 4k. You don't want a single write straddling the end of FAT1 and the start of FAT2; neither do you want a single 4K cluster write to straddle two sectors on the SSD itself.
    – Joshua
    Feb 22, 2023 at 14:36
  • 1
    I made a NASM script to create a FAT FS image, either FAT12, FAT16, or FAT32 (including small FAT32), and optionally including an MBR as well as also optionally a dosemu image header. It's at hg.pushbx.org/ecm/bootimg -- I do have options to change the MBR gap size as well as to alter the reserved sectors amount to make the cluster data blocks aligned, though I did not yet consider aligning the FATs themselves. Good idea, maybe I will get around to adding that!
    – ecm
    Feb 22, 2023 at 14:59
  • 1
    I don't make an answer as another answer has determined the earliest DOS which supports more than 1 as Reserved Sector value. But Microsoft warns that other values than 1 are not universally supported, and you don't need to change it. Just have the partition start at e.g. sector #7 or #63 in LBA (which is not sector #7 in CHS but #8), and the next sector, FAT starts 4K aligned. Then FAT size must be 8/16 sectors for 4k/8k size per FAT. Then the Root Directory entries must be 256/512 for 8k/16k size. Then the cluster size must be 4k or 8 sectors. But SSD wear leveling should make this useless.
    – Justme
    Feb 22, 2023 at 21:53

2 Answers 2

17

Since I just so happen to have a battery of floppy images with various DOS kernels I compiled to investigate a somewhat similar problem, I decided to try the probing route. I did not use the image in the question, though; instead, I created the disk image attached below (also bzip2-compressed), geometry 32 × 16 × 32, and used that for testing.

begin-base64 644 hda.img.bz2
QlpoOTFBWSZTWRIY2P8BAU5/ffRWwADAOXAQsyuPAAIAUABAAgiQACIAEEAI
AEmwANmzDTSob1QyA0DQPSBmo0G9SDRCGjQnqYCZA0GAmICKSg0AAANAaAAH
yrDGOQn25IB7wIpbIUFlDYhUAVVZqyFEExCIAgAFeBswiqgAAxC51JMnTbwu
SSjJpggAAQAAAfUgmGUD7yE8VzvTcJmTETsUKNZTgRCrte8NEsYNS19xXVXN
hksyRjK8s+n1JqeeCfSbc7mxwo+WhR+Id1y+VQIiIgAAAPxQHI/i7kinChIC
Qxsf4A==
====

The partition was formatted with mkfs.fat 4.2 and then manually edited in a hex editor to use a more compatible OEM ID and scrub the boot error message. I mounted the above image as a virtual disk in QEMU, and the test kernel image in the floppy drive. I booted the kernel from the floppy and then ran type c:test at the command prompt. If the command succeeded, I presumed the kernel reads the drive correctly.

All the DOS kernels I tested detected the partition, but the earliest version of MS-DOS that succeeded in reading the file was MS-DOS 5.0; MS-DOS 2.0 through 4.0 could not find it. PC DOS was similar, unsurprisingly, as the two were still different builds of the same kernel back then: 5.0 worked, 4.01 did not. The file was also read successfully in DR DOS as early as version 3.40. Also unsurprisingly, the file reads correctly in FreeDOS.

With the versions that failed to read the file, I also tried writing to the disk image, which revealed an interesting divergence: PC DOS 2.0 not only failed to account for more than one reserved sector declared in the BPB, it was apparently also convinced that the FAT has only 6 sectors instead of 12 as the BPB declared. It also had no problems allocating cluster #1 to the newly-created file, even though cluster numbering starts from 2. (That’s right: cluster #1 overlaps the root directory area.) The later MS-DOS 3.30 through 4.0 only have the former flaw.

14

This smelled like a problem with the two operating systems disagreeing on where the filesystem actually started because they are exactly a sector different in where they think the metadata is, and I suspected that the difference was between the CHS and LBA information in the partition table. So let's look at it:

$ fdisk disk.img
Disk: disk.img  geometry: 160/4/63 [40320 sectors]
Signature: 0xAA55
         Starting       Ending
 #: id  cyl  hd sec -  cyl  hd sec [     start -       size]
------------------------------------------------------------------------
*1: 01    0   0   7 - 1023 255  63 [         6 -      32666] DOS FAT-12
 2: 00    0   0   0 -    0   0   0 [         0 -          0] unused
 3: 00    0   0   0 -    0   0   0 [         0 -          0] unused
 4: 00    0   0   0 -    0   0   0 [         0 -          0] unused

The partition start passes muster: CHS (0,0,7) is LBA 6. The end looks very wrong because CHS (1023,255,63) is bigger than the geometry, but that's not actually the problem here.

Per e.g. §6.2 of Large Disk HOWTO, "A well-known claim says that partitions should start and end at cylinder boundaries." and "MSDOS 6.22 has an alignment requirement. Extended partition sectors that are not on a cylinder boundary are ignored by its FDISK. The system itself is happy with any alignment, but interprets relative starting addresses as if relative to an aligned address: The starting address of a logical partition is given relative not to the address of the extended partition sector that describes it, but relative to the start of the cylinder that contains that sector".

The detail is not quite as I remember and not entirely clear either, but the "partitions should start and end at cylinder boundaries" certainly is. Your partition table says it starts on sector 7, but DOS is (sometimes?) not paying attention to that field.

Meanwhile, modern hard disks use 4 KiB sectors, and although SSDs don't actually need to use a specific sector size because they use compression and other trickery so the logical sectors do not align with their internal storage anyway, they are intended as drop-in replacements for hard disks and pretend to be 512e for maximum compatibility. So you should certainly design your partition layout to ensure 4kiB alignment (rather than the 1kiB in your question), which means a multiple of 8 sectors.

To satisfy both DOS and modern disks' requirements, you need to find a partition start (c, h, 1) which maps to a multiple-of-8 LBA. Because there are usually 63 sectors per track, you are looking for a (c * head-count + h) which is divisible by 8. Thus the first suitable CHS is (0,8,1) which corresponds to an LBA of 504.

14
  • 3
    @Justme: Don't trust fdisk. When you load a disk image into your VM it's going to invent geometry. And we've already established that the MBR end is broken. So this leaves boot sector geometry, for which you found the correct value.
    – Joshua
    Feb 22, 2023 at 14:41
  • 1
    Oh fun. Is the documentation about DOS looking for a SHORT JMP instruction at the start of the boot sector just wrong?
    – Joshua
    Feb 22, 2023 at 15:03
  • 1
    @Joshua DOS Internals, which I have at home, does say DOS may check the jump. I cited it in hg.pushbx.org/ecm/ldosboot/file/a4823a5555d4/boot.asm#l506
    – ecm
    Feb 22, 2023 at 15:09
  • 2
    @ecm True, technically there can be just any arbitrary executable code if DOS compatibility is not needed. But for FAT driver compatibility, many storage formats such as SD cards have specified that the FAT boot sector must start with EBh, xx, 90h or E9h, xx, xx.
    – Justme
    Feb 22, 2023 at 18:04
  • 1
    I think the answer is important, but from a different aspect. The original problem appears not to be directly related to the FAT partition having non-cylinder aligned start address, but the fact that the used MS-DOS 2.0 does not understand the FAT filesystem resedved sector count at offset 0x0E. In Microsoft documentation for FAT file systems, they explicitly warn about using value other than 1 here, and it seems MS-DOS 2.0 assumes it to be 1 without checking. Using a value of e.g. 2 is supported correctly under MS-DOS 6.22.
    – Justme
    Feb 22, 2023 at 20:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .