Retrocomputing Stack Exchange is a question and answer site for vintage-computer hobbyists interested in restoring, preserving, and using the classic computer and gaming systems of yesteryear. It only takes a minute to sign up.
Sign up to join this community
Continuing my nostalgia reading of Dan Gookin's DOS For Dummies, there is a section that literally advises against compression programs like DriveSpace and calls it a solution to a problem not some glorious road that one must take.
On more recent operating systems, compression is not something dangerous to the best of my knowledge. So what was the dangerous part?
The dangerous part was that your disk contents would be compressed.
i.e., you have data that can only be read by the compression engine.
If the compression engine was buggy, no data for you. If other buggy software corrupted the container file, no data for you.
If you had disk problems or file system corruption, you'd be limited to recovery tools that incorporated the same decompression algorithms. Contrast that to the fairly wide-open landscape of plain FAT tools.
The main problem was that DOS drive compression programs such as DriveSpace, Stacker, or SuperStor created a virtual drive and stored its contents in a large file on the original drive.
This created two risks:
Errors would generally result in the loss of multiple files, or even the whole volume, and were difficult if not impossible to repair.
Modern file systems supporting compression and write access tend to compress files individually inside a regularly-structured file system, reducing the risk.
In addition to the compatibility problems and the data recovery problems mentioned in the other answers, there is one inherent problem with the disk compression schemes:
They are sometimes quite off from the efficiency promise they offer.
The promise is that the added time for decompressing a file is somewhat offset by the reduced time for reading the now-smaller file from disk. One can even expect an overall speedup if the disk is slow and the CPU is fast. So far, so good.
On the other hand, if one tries random access to a file (i.e. not reading it as a whole from the start to the end) the promise somewhat breaks apart.
Writing is not quite as fast either. In contrast, for a non-compressed filesystem, writing is almost as fast as reading.
And things go really, really bad if one tries random-writing in a file (e.g. a database-like workload). The usual result is that the random write in a file is either impressively slow (100x is not unusual), or the file grows larger than the real data contained inside, or both, depending on the particular compression scheme used.
Then, there is the issue of completely arbitrary reporting of the free space available. Some software actually used these numbers and failed when they were inflated too much.
I'll add some context to the good technical answers already given.
When the first on-the-fly disk compression software for PCs (Stacker) came out, it had a huge impact. It became prevalent faster than any software I'd ever seen, or would see. Storage was expensive, and this software was manna from heaven.
This even reached to point where computer ads started quoting machines with 80MB HDDs and "160MB" machines, machines with 100MB HDDs as "200MB" machines, and so on. This was circa 1992, I think, and you could flip through hundreds of pages of Computer Shopper magazine without seeing an ad that didn't use this trick.
I think the book you're quoting is (quite reasonably) saying "slow down." Yes, Stacker was an amazing program, but there are tradeoffs involved. It's a gentle reminder to an MS-DOS user community that very much needed it at the time, and you wouldn't know that unless you'd lived through it.
There was an issue with using it for floppies. (You remember those, right? Weird shaped thin packages that you stuck in a slot on your computer to carry a small amount of data from one place to another. This was before such things as WiFi and so on. How primitive!)
You would get used to how long it took to write the data to the floppy. And you would wait that long. And the active light would go out. So you would pull the floppy out to head to the other machine. But the reason the light went out was because it had timed out waiting for the compression program to finish compressing the rest of the file. It still had a lot to write. Now the write has failed. Your floppy is corrupted. If you put the floppy back in, it probably does not help.
Even better, the error message is sitting there on the source machine. Unless you stood around and waited for the compression program to start to write again, and have it time out, you might not notice your floppy was corrupted. You might get all the way to the client's office with a corrupted floppy. Since you were in a hurry to get the data to the client by the deadline, probably you were not waiting around for this.
The corrupted floppy would sometimes be resistant to being uncorrupted. The corrupted file might, or might not, just let you delete it. You might need to reformat the floppy to get the media back.
After a few cycles of this kind, the company decided not to apply compression to floppies.
In those times Norton Disc Doctor could handle & fix corruptions of most disk compression algorithms (being on filesystem level and volume level).
Some compression systems used a file which contained the whole virtual volume, other systems made a new partition with the partition type set to some identifier which wasn't used in that time.
Some problems could occur, or the virtual volume became corrupted, or the identifier in the partition table became corrupted. Or worse the whole partition table became corrupted.
I remember one case recovering such a compressed "disk" completely low level hex editing a partition table because the partition table was corrupted and the compressed partition was gone.
On the other hand, due the nature of compression and depending on the algorithm used it was sometimes possible to recover (parts of) those volumes depending on the redundant info that was contained in the volume. Some compression schemes contained redundant info to make it less error prone. (One could compare it with the technique used in some raid systems where redundant info is contained on disk. If one disk fails one can rebuild the raid from scratch)
Compressed drives also require more work to use/own.
When a drive is compressed, the host drive contains a CONFIG.SYS, AUTOEXEC.BAT, a container file that holds the compressed user data and any other files referenced by CONFIG.SYS and AUTOEXEC.BAT. While booting, it's these CONFIG.SYS and AUTOEXEC.BAT files that are used. After the compression driver is loaded by CONFIG.SYS, it's assigned a new drive (e.g. D:) exposing the data in the container by compressing writes and decompressing reads for that drive. After mounting the container file, it swaps it with the original host drive letter (what was D: become C: and visa-versa). The problem is that when you install new software or make a configuration change, the CONFIG.SYS and AUTOEXEC.BAT files in the container file are updated and not the ones used during the boot. It requires the files on the boot drive to be synchronized with the copies in the compressed container.
Compression amplifies the problem of fragmentation. A normal FAT volume allocates in fixed size clusters. Compressed data is allocated using variably sized blocks with the granularity of a sector (or smaller, Stacker 4.0 could pack multiple small bocks into a single sector). Fragmentation is at the sector level and not just the cluster level. Without regular defragmentation, even small files could require multiple reads with data spread across different tracks undoing the gains from the smaller (compressed) i/o.
