Filesystems with versioning

Question

I've been reading through The Unix Hater's Handbook. It has many, many very valid criticisms. (I'm still raging that terminal escape codes aren't in the terminal driver...)

There is one anomaly though: One of the chapters is complaining that Unix doesn't support "file versioning", when "real operating systems" have had this feature for years.

Now they don't really elaborate on what that means, but it seems to mean a system where each time you save a file, the old version is still kept, and the new revision just gets a new version number.

I am not personally aware of any operating system in the entire history of computing ever having had this feature. Can someone enlighten me as to what these mythical systems were?

Unix was written an extremely long time ago. As I understand it, in Those Days a "large" system might have as much as 2 kilowords of memory and presumably a similarly tiny amount of disk. The "real operating systems" the book alludes to would presumably be even older than Unix, and hence for even more constrained hardware.

I find it very hard to believe that a system with a 4 Kword disk would "waste" disk space by keeping every prior version of every file ever created. That just seems like you'd run out of disk space within ten minutes.

Have I misunderstood what they're talking about? Or were there actually systems that worked like that?

Answer 1

I am not personally aware of any operating system in the entire history of computing ever having had this feature.

Siemens BS2000 of the early 1970s may be an example here (*1) with a feature they called file generations. A new file could be marked in the catalogue as having generations, setting a base generation number and how many generations are to be held (*2). It was presented by a single entry, and for most purpose handled like any other file.

To address any of the generations a file name could be suffixed with it's generation number. For example a file named "TEST.FILE" could be defined as holding up to 5 generations with generation 6..10 existing and generation 10 being the newest (actual). Valid names for file operations would be:

• "TEST.FILE" accesses the actual generation (#10)
• "TEST.FILE(9)" accesses the explicit generation #9
• "TEST.FILE(-1)" also accesses the generation before the actual (as well #9)

The actual generation (pointer) could be moved using shell commands and/or an API. For example to revert to a previous version. If in our example it would be set to 8, then

• "TEST.FILE" accesses the actual generation (#8)
• "TEST.FILE(-1)" also accesses the generation before the actual (#7)
• "TEST.FILE(+1)" also accesses the generation after the actual (#9)

This mechanism makes it easy to handle things like logfiles, program versions or databases. Roll-back or roll-forward can be done by a simple command and unlike any naming scheme, no program has to be modified to work with file generations - that is, unless some special features are to be used - all they see is a regular file.

Unix was written an extremely long time ago. As I understand it, in Those Days a "large" system might have as much as 2 Kwords of memory, [...]

Erm, these were the smallest systems. Keep in mind, a PDP of that time was the lowest end of computer available. The upper end, where 'real' OSes were used, was quite different. For example, the database system used for the 1972 summer Olympics used two mainframes with 2 MiB of core memory each and more than 30 disk drives of 77 MiB each (*3). Those were large systems (*4). Not PDPs.

Admittedly, such a configuration is close to the upper limit what was used at the time, but it wasn't a unique installation.

I find it very hard to believe that a system with a 4 Kword disk would "waste" disk space by keeping every prior version of every file ever created. That just seems like you'd run out of disk space inside of ten minutes.

Like with many other features, they are only useful on a capable setup, the same way that subdirectories only make sense with drives large enough to hold them and so on. Equally important, features only make sense from an application viewpoint. An application needing do hold versions will love an OS that supports it in a consistent way.

Bottom line: Developing an OS's capabilities orientated at the smallest possible configuration doesn't sound like a good idea, or does it?

Have I misunderstood what they're talking about? Or were there actually systems that worked like that?

Quite a lot. In the mid '70s it was seen as a great addition to extend the usability of file systems. At that time many features we would nowadays place request from a database system were provided directly by OS and file system.

Oh, and it's not only a thing of the past. IBM zOS for example does as well support their mechanic of file versions, called a Generation Data Group.

---

*1 - BS2000 was based on RCA's TSOS, but I'm not sure how much was already present in TSOS.

*2 - This includes tape storage, so generations could be moved to tape for long term storage (and to save disk space). The catalogue would still be used to manage them.

*3 - Yes, that's in total about 2 GiB in 1972 :))

*4 - That view, users of 'real' computers had on Unix is reflected in the Unix Hater's Handbook, isn't it?

Answer 2

FILES-11 on DEC minicomputers was a versioned file system -- RSX-11M, IAS (on PDP-11), VMS (on VAX, Alpha).

Version numbers are very user-visible; they are part of the syntax for specifying a file. And programs are designed to behave appropriately for a versioned file system.

When creating a file, the normal way was to not specify a version number, and the system would assign one higher than the highest extant version. This is the "normal" approach for editors and similar file-modifying programs. Much easier than juggling .BAK files, etc.

When opening a file, the normal way was to not specify a version, and the system would open the highest extant version. This is the normal approach for using programs that just read file. If the user is typing in the name of the file to be opened, they can specify a version or not, as required.

You could specify a version, which allows you to modify a file in-place if desired (when writing) or read any previous version. That'd be normal for a file that, for example, was used for random access (database, ....)

With respect to "not running out of space" -- two things. Firstly, typical disks of the time could hold many thousands of files with typical file sizes of the time. Think program source files (Macro-11). A large file is, what, 1000 lines? That's surely under 50K bytes, or 100 blocks in PDP-11 terms. An RP04 disk pack, a storage device from around 1974, held about 88MB. For another data point, the RSX-11M-PLUS kernel (exec and drivers) source files occupy around 4.5MB on my PiDP-11 system.

Secondly, people generally tidied up. While doing active program development you'd likely end up with dozens of versions. When happy you didn't need to go backwards, you'd purge down to one version. And the computer operator might very well decide to purge everything down to a couple of versions (if he was nice, with fair warning to the users) if the disks were getting near full. In summary, users were aware they were using a finite resource and behaved accordingly.

In my opinion it's a giant step backwards to not have a versioning file system.

Answer 3

There were quite a few operating systems that had file versioning in the same era as unix.

Many file systems that we are familiar with today just have some components of a file name, such as:

Name.type

They might have a path:

\folder\folder\Name.type

They might have a server (UNC as an example):

\\server.domain\folder\folder\name.type

In many current systems, if you make a copy of the file, or attempt to overwrite with the same name, it might asks if you want to overwrite. If you choose not to you get a version number appearing:

Name.type(1)
Name.type(2)

So you can experience file versioning on current operating systems.

However, a fully versioned file system would not normally overwrite a file. It makes a new version number every time. This means whenever you use an editor or save a spreadsheet, or other document you get a whole stack of numbered versions of the file. The version number is usually stored at a distinct location in the directory structure and is not part of the name. If you refer to a file by name you are given the latest. You can clean up old versions of the file with specific system commands, like PURGE.

Two example operating systems that used this were VMS (from DEC) and George III (from ICL). (I can probably add a fair few more when my memory digs them out).

Did it exhaust storage: yes and no. The file storage as usually quota'd to users (which were always multi-user) and each individual user may exceed their quota and have to tidy up. The other aspect is that most files were just text. There was much less multi-media like images and video that we have today. The other thing is that these computers were huge, and not as small as you imagine, and that exchangeable disk/tape storage was the common way of having many files - you swapped the disc pack and the tape.

Answer 4

In addition to what others wrote: ITS, TENEX, TOPS-20.

In ITS, files are named by two strings each at most six characters. The second file name can be a number to specify a version. If you open a file for reading, > will access the latest version. When writing, it creates a new version. < refers to the oldest version.

Moby edit. Let's make a timeline.

• ~1965 - Project MAC: MACDMP
• 1967 - Project MAC: ITS
• 1969 - BBN: TENEX (→ DEC TOPS-20)
• 1969 - ICL: GEORGE III
• 1971 - DEC: RSX-11 (→ IAS, VMS)
• ~1971? - Siemens: BS2000
• 1993 - Microsoft: Windows NT

Answer 5

My experience is with the VAX and VMS. It had versioned files.

Back in the day, it was not uncommon for some programs, like editors, to create a backup copy of the file you were working on. In the end you'd have, for example, file.txt and file.bak.

The versioned file system is simply that concept writ large. Instead of file.txt and file.bak, you had file.txt;2 and file.txt;1, with the lower numbered version being the older one.

You'll note that it's not used for files that are changed in place (notably, things like databases). Rather they're for files that are rewritten wholesale.

If you open a file for writing that already exists, rather than overwriting the old version, it simply creates a new version. It's a simple mechanic.

On systems like UNIX, the applications have to jump through hoops to manage this. .BAK files, adding time stamps to file names, file_2.txt, etc. On versioned files systems, this is unnecessary, and "free" for all applications.

VMS has a PURGE command that goes through and removes all of the older versions.

It should be noted that modern macOS applications implicitly version files today. The OS has built in application support for this model (note, the filesystem does not, the application framework does). Edit, for example, a word processor document, and it internally makes new versions and manages that for you.

This is distinct from modern version control systems, which, obviously, also do this but offer a different workflow from versioned file systems. Many developers leverage these version control systems not just for source code, but many different files.

I believe the Symbolic Lisp Machines used a versions filesystem as well.