Did any software use "deleted data" floppy sectors?
Yes. A lot.
... but it's complex, not at least as this isn't a software and especially not an application side issue but hardware and hardware-like as well.
The standard IBM System 34 floppy format supports two different types of sectors, "data" and "deleted data".
That definition is way older and came already with the basic 33FD format of 1973 (*1) used with the 3740 Data Entry System. What System /34 introduced were 53FD drives using DS DD (MFM).
So, yeah, lets go back way, way, way back, right to the creation of the world ... err, the floppy as storage media (*1).
In the sector header, [..] deleted data sectors by an F8 byte (Deleted Data Mark aka Deleted Data Address Mark or DDAM).
Sounds like wording from a certain PoV (*2). In basic floppy documentation this is called a Control Mark (CM), as it marked the sector as one containing control information. The 765 bit recording detection of an F8 address mark (ST2/D6) is accordingly named Control Mark. It's the basic way of Out-Of-Band-Signalling (*3): Sectors with FB build the data stream, that is all data content is user data, while F8 sectors hold meta information about how to handle/find the data stream.
What type of Control Field the data of that sector represented was encoded in the first data byte(s). IIRC there were several different use cases, all marked by
Deletion - letter
D (x'C4' (*4))
At least with 3740 compatible handling the remaining bytes were unchanged, allowing to 'undelete' a record later on (*5).
Unused Label -
D as with deleted
The first cylinder of a floppy is reserved for IPL and label (*6) information All sectors not used for label
Inserted (but not used) -
D followed by Backslash
\ and Spaces (x'C4E04040...')
Some 374x allowed not only add, edit and deletion of records but as well inserting. This was done moving up all sectors by the number of sectors to be inserted and filling all 'freed' ones with
D\ and spaces.
D as with deleted
An overflow control fiels told the controller to look for the next sector using the overflow area.... at that point memory gets blurry - could be that this wasn't with the 3740, but somewhat later)
Sequential Bad Sector -
Mark single bad sectors. The following sector will contain the data.
Relocated Bad Sector -
Marks a bad sector, the replacement is to be found by looking track/sector number up in the bad sector list on Track 0. Usually the last two tracks of a disk were reserved for relocation.
Bad Cylinder - Done by ID field
If a wohle cylinder is to be marked as bad all it's ID fields will contain the number FF.
The last isn't about a CM, but part of the same mechanic, as all of that was made to have the disk controller act independent from the system (CPU) when it comes to reading a given sector. After all, it simplifies error handling a lot when standard cases are already done by the hardware. To understand this one must note that the 3740 station had two discrete (*7) processors:
- The main processor handling input, editing, display, etc. and executing the record program (*8)
- The floppy processor handling the drive.
And, depending on mode, it is the floppy processor that does handle most of the above during read independent of the main processor.
When in sequential read a CM is encountered, the first byte is checked as well. If it's a
F, reading will continue with the next sector as if the last has not been seen at all. In all modes an
F will result in a continued read with the next sector. Same it's a
.. Here the bad block list will be read and the replacement sector will be fetched. Same and independent of AM, if an ID x'FF' is found the head is advanced by one track and then looking again.
All of that enables rather simple operation at the main processor side as the most common cases are handled invisible (*9). Only if he cares to see deleted records (
D) must be told - which is only the case in some editing modes. Like restore or insert.
All of his was set in stone long before the first mini computer - and even less micro - or integrated floppy controller touched a floppy disk.
Everything that came after was either using that standard in full or part or misusing it to reach individual goals (like copy protection). Wich of course may be as colorful as the human mind is when trying to come up with new applications :))
Many floppy disk controllers have separate commands for dealing with the two different sector types. The NEC uPD765 [...] and its compatible successors (Intel 82072, etc)
If looking for original handling, one should best start with WD 177x/9x series, as NEC's uPD765 is a later development.
have READ DATA/WRITE DATA commands to read/write data sectors, and READ DELETED DATA/WRITE DELETED DATA commands to read/write deleted data sectors.
Well, where the WD 17xx offer only basic ability of address mark handling(detecting i, writing it), the NEC 765 does implement the whole continuous read/skip mechanic IBM had in mind for CM. With the 'normal' Read Data command and the SK-Bit (D5 in the first command word) set to 1 the 765 will behave quite like a 3740 in sequential read: A sector found with an CM will not return an error but be skipped and the next following sector delivered instead. Read Deleted Data inverts that behaviour. Handy, isn't it?
My question is–did any software actually use "deleted sectors"? What did they use them for?
Tons. Starting with any software on minis or micros that had to read IBM diskettes for data transfer. After all, connecting to big iron or (god help) trying to replace it won't work without handling the interfaces it's to be linked in.
OpenVMS has an API for [...it...] but that still doesn't make clear whether any software actually used that API and what it used it for.
See above. DEC was the new comer to the show - without handling existing interfaces it would be an island noone would ever buy (*10). In fact did DEC support 3740 as early as 1975 with their first Drive (RX01) for PDP-8 and PDP-11 systems using the RX8/RX11 controllers.
While DEC operating systems did not use a CM in regular operation, the controllers were specially fitted to detect and produce a CM. Standard 3740 having deleted records by default makes support for reading as well as writing mandatory for any kind of data transfer using diskettes to and from IBM mainframes.
The use of low priced diskettes instead of tapes for small to mid sized data sets opened new markets for DEC - many departments liked the idea of buying lower priced DEC systems instead of renting 374x while at the same time improving data entry by using the more capable way of programming.
Same goes for engineering departments in companies and universities now able to move their data sets direct to mainframe without the need of expensive communication lines or tapes.
I've heard some suggestions they might have been used to mark bad sectors (although I don't think that was the original intention behind them).
It has and it was. Not just on floppies but disk as well - that's where it originated.
Minicomputer and mainframe systems (with record-oriented file systems) sometimes support marking individual records within a file as "deleted", much like deleting a row from a database.
Record, not row. Think punch cards, not databases (*11).
I suspect that was what IBM was thinking when they introduced this feature, but I'm sceptical it ever saw much use for its intended purpose.
Don't tell millions of users of mainframes that did use those features direct (like with 3740) or without knowing.
I've also heard suggestions it has been used for copy protection.
I would classify that as a misuse(*12). It's based on the fact that 765 read command is usually issued without SKIP (SK=0) resulting in an abortion standard disk copy software does not handle, doing he desired protection. Likewise with SKIP (SK=1), that sector will be skipped and the next delivered instead, so 'automatically' corrupting the copy by leaving out the one sector.
One other use I've discovered is that LS-DOS (for the TRS-80) marked all the sectors on the directory cylinder as "deleted data". Possibly this was done to discourage user software from modifying it;
Sounds like a great way to protect unintended alteration. Kind of a read lock - and a fast detection of corruption.
I suppose it also could have played a role in locating the directory cylinder (if it was in a non-standard location) although I don't know if it was ever actually used in that way.
Rather not, as it would mean one had to read the whole disk to find them.
*1 - If not with the 1971 23FD, but I'm not sure and didn't find any 23FD record definition with that detail.
*2 - Or 'grown' usage as the 3741 and 3742 stations allowed programming to keep the data of a deleted record untouched. Only the address mark was changed to F8. Those disk were of course incompatible with basic 3740 or a direct attached unit.
*3 - In-Band-Signalling would be if control information is embedded within the user data stream - like using escape sequenced. In band signalling has the advantage of less prerequisites (i.e. media definition) but the disadvantage of more complex data handling.
*4 - All encoding is in EBCDIC. Those systems were at foremost meant to replace punch card stations for main frames, thus all markings in mainframe code. This stayed true even with later systems using ASCII for data. Another advantage of Out-Of-Band-Signalling: Control encoding is complete Independent of any user data.
*5 - The missing character had to be entered again. To turn this into and advantage experienced designers of data entry schemata usually made the first byte a type indicator of the record, so anyone editing had less of a problem to reconstruct that even a long time later.
*6 - Label is IBM terminology for any kind of descriptive data, like disk geometry, block lists, directories or alike.
*7 - Discrete as in independent as well as in build from discrete components, not microprocessors or alike. Remember, the 3740 was introduced and delivered January 1973.
*8 - Don't get me going on that. It was horrible and brilliant in one. horrible in how to encode it and what all was not possible, but brilliant in what could be done with that wired stuff in terms of data entry process.
*9 - Doesn't sound different from Hard disks we know, does it? And in fact, the whole basic operation is copied from disk handling before the FD was invented. So nothing new here :))
*10 - At least not the ones that controlled the budget - _'Is it IBM compatible' was a notorious question way before the PC came along. IMHO unconscious transfer of that deep seated preconception was what made the PC what it became.
*11 - Not to mention that simple measures always beat complex ones :))
*12 - Then again, one could argue it's a valid use of a control field supplying information important to handle the data stream of that game :))