3

On this website (and others, of course) you could be informed that there is a special partition flag (“hidden”) to avoid detection by Microsoft operating systems.

First: Is this a convention by/with MS or a trick?

Second: How could a Microsoft OS be so dangerous to others’ partitions? Was there an advantage hidden in that?

Third: When and why is this not used today in modern systems?

Ok... The questions are somewhat rare and strange. But I think they have a potential to clarify some things of past and today.

1
  • 3
    A current partitioning scheme and modern operating systems is not really retrocomputing. – JdeBP Oct 28 '20 at 1:50
7

GNU Parted's flags are misleading.

GNU Parted's "flags" are not accurate representations of how partition tables actually work. They are abstractions, that are mapped, with varying degrees of congruence, onto the actual partition table mechanisms.

  • The MBR-style (a.k.a. "DOS") partitioning scheme originally had an 8-bit type code and 8 bits of flags (only one of which was actually used) for each partition table entry, which IBM extended to have some more flags in an ancillary table in an adjoining disc block.
  • The EFI partitioning scheme has a GUID for the partition type and a set of 64 attribute bits for each partition table entry. 48 of the attribute bits apply in general to all types of partitions and 16 of them have meanings that depend from the partition type.

Parted's hidden flag does not actually map to a single real flag bit in the MBR-style partition table, and maps to an unexpected flag in the EFI partitioning scheme.

In the EFI partitioning scheme, one would expect Parted's hidden flag to map to the "hidden" attribute bit. The "hidden" attribute bit is one of the type-specific bits, bit 62, and only means that for Microsoft Basic Data partitions (type EBD0A0A2-B9E5-4433-87C0-68B6B72699C7). But Parted's hidden maps to the (general) "the platform requires this partition in order to function" flag instead. (Ironically, this is more akin to "system" rather than "hidden".)

In the MBR-style partitioning scheme, Parted's hidden flag alters the partition type code. It does not correspond to a flag or an attribute bit in the actual partition table at all.

Various tools have made partitions "hidden" by altering their type codes to ones that operating systems do not recognize. Andries Brouwer's list of partition types as of 2015 listed IBM's Boot Manager, FreeDOS's fdisk program, Acronis's OS Selector, and BlueSky's Power Boot each with different formulae for altering the types of "hidden" partitions. (This is a deceptively short list; as other softwares have simply adopted the mechanisms of one of these tools.)

Parted only knows about the IBM scheme, which has a parallel set of partition type codes (for the few "hideable" types of partitions) with bit 5 toggled. But this is far from being an IBM or OS/2 thing, nor even was it three decades ago.

What partition type codes (in the MBR-style scheme) do

Partition type codes prevent operating systems from trying to automatically mount disc volumes that they either won't recognize or won't handle correctly. Operating systems such as MS/PC/DR-DOS and OS/2 would scan the entire partition table and try to automatically mount disc volumes for every partition that had a type code that they recognized.

The various "FAT" partition types are widely mis-named. They didn't denote 12-bit, 16-bit, or 32-bit File Allocation Table widths. They denoted partitions that earlier versions of MS-DOS would handle wrongly if they tried to use them, for various reasons: they went over a 16MiB, 32MiB, or 2GiB size limit in the older operating systems' code; they were positioned across or beyond the limit of Cylinder+Head+Sector addressing in the ATA disc I/O command interface; and so forth.

What "hiding" really was

IBM's tools (which were not strictly speaking IBM's Boot Manager, as IBM's fdisk, setboot, and to an extent lvm commands were also in on the act) were also stopping operating systems from automatically recognizing disc volumes, but for a slightly different reason. Rather than preventing old code and old addressing modes in I/O commands from breaking on bigger volume sizes and larger discs, IBM's tools were trying to get operating systems to correctly recognize their own boot volumes.

(I am using the Microsoft "boot volume"/"system volume" terminology in this answer: The "boot" volume is the disc volume containing the main part of the operating system, all of its data files, utility programs, and so forth; whereas the "system" volume is the EFI System Partition or similar.)

This wasn't about old operating systems being dangerous to the hidden volumes, it was about old operating systems not being fully able to identify which disc partitions were themselves.

Some old operating systems would simply assume that they were bootstrapped from the first primary partition in the partition table that had a recognizable partition type, and take that to be their boot volume. Newer operating systems would do things such as: check the in-memory copy of the Volume Boot Record for its BIOS Parameter Block, and compare the sector start position to the various partition table entries to see which one was the boot volume; or have a bespoke boot manager that encoded the boot volume location as an ARC Path that it passed along to the boot loader program; or patch the boot volume location into the kernel image. But a few old ones simply made a simplistic assumption.

IBM's tools allowed selecting any primary partition to boot from, so it hid and un-hid primary partitions so that only the right one, actually selected by the user at bootstrap or preselected by the setboot command, would be recognized as the operating system's own boot volume if it were one of these old operating systems.

IBM's tools were not alone. There were a number of boot managers and partition utilities around two to three decades ago, some of them lost to history, and some of them switched things around like this (or in other ways) too.

If you look up Jan van Wijk's DFSee — still going after 25 years — you will see that its setaccess and startable commands do this sort of thing as well.

This mechanism was not vital.

The necessity for this has since gone away.

The EFI partitioning scheme only has primary partitions, and has forced the hands of operating systems that were assuming that there was only ever going to be one recognizable primary partition and it was them. The EFI Boot Manager and other boot managers can now pass EFI device paths and other sorts of information into boot loader programs.

It wouldn't have really been necessary even back then, in fact. There was a protocol that worked even back then.

As mentioned, the in-memory copy of the Volume Boot Record contained the BIOS Parameter Block for the volume, which had a "hidden sectors" field indicating where on the disc volume it was located. This location information could be used as-is in the case of primary partitions, to match against the partition table entries to determine which partition was the boot volume. Boot managers, both IBM's and other people's, would fix up the in-memory BPB for secondary partitions, whose on-disc values were relative rather than absolute positions; so that the matching procedure would work for them as well. An operating system simply looked through the partition table until it found a partition with the same start position.

As mentioned, though, not all operating systems were this smart. OS/2 was, and being able to find its own boot volume correctly is what made it possible to install it entirely on secondary partitions (albeit that its installer was still dumb enough to think that only IBM's Boot Manager would do the BPB fixups properly, and require that IBM Boot Manager be present); but others were not.

Further reading

7

MBR partition tables have no 'flags', with one exception: the value at offset 0 that indicates the active or bootable partition.

What they have is a partition type indicator, a byte value (0 to 255).

DOS recognized certain partition types as containing file systems it knew about (FAT, FAT16, FAT32). If you changed the type to something else, DOS would not recognize it, i.e., it would be 'hidden' from DOS.

See list of partition types. It's claimed there that the OS/2 boot manager used to hide partitions except the currently booted system. It did this by replacing the 0x4 type by 0x14, and so on. So now we say that if 0x4 denotes FAT, 0x14 is a 'hidden' FAT partition. But at base it's just a partition type that DOS does not recognize.

The point, I suppose, was to ensure that a given DOS instance was isolated (when running) from other instances.

Whether Microsoft or IBM was responsible for implementing that, I don't know.

The same thing could be done in principle for any other OS's file system types. To "hide" a partition from an OS, change the partition type to some number that the OS does not use or recognize. Of course, that needs the OS to leave alone partitions that it does not recognize, rather than doing something destructive to them.

That there are no partition types reserved for "hidden" variants of other standard types is just a matter of need. No-one had any reason to want them, I assume.

GUID partition tables, by contrast, have an attributes/flags quadword. Microsof defines a 'hidden' flag at bit 60.

On Windows 10

DISKPART> detail part

Partition 2
Type    : e3c9e316-0b5c-4db8-817d-f92df00215ae
Hidden  : Yes
Required: No
Attrib  : 0X8000000000000000
Offset in Bytes: 525336576

There's nothing particularly mysterious about this. Sometimes operating systems think that some things are not generally interesting, will clutter output, etc., and therefore define a mechanism to tag things as 'hidden'.

It's not different in concept to, say, the fact that the Unix "ls" command does not, by default, show files whose names start with a dot.

4
  • Really nothing special!? Ouch... – Daniel Bandeira Oct 27 '20 at 23:38
  • But... Why is there no flag "hidden" to other operations system? They don't clutter about partition flags? – Daniel Bandeira Oct 28 '20 at 0:13
  • 1
    I rewrote/rearranged my answer, is that clearer? – another-dave Oct 28 '20 at 2:41
  • Q1: Yes, it's a trick (for OS/2). Q2: In general case, it isn't dangerous, but it is a good idea avoiding OS that don't use such partitions finding them! Q3: "Hidden" flag is used yet, but in a proper (to GPT) and conventionalized form (by Microsoft). – Daniel Bandeira Oct 28 '20 at 10:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.