How should we interpret Dave Cutler's criticism of Unix?

Question

Dave Cutler is well known for his contributions to operating systems, having led the effort on VAX VMS at DEC and Windows NT at Microsoft. According to his Wikipedia page, he is also known for his attitudes toward Unix.

[Cutler] expressed his low opinion of the Unix process input/output model by reciting "Get a byte, get a byte, get a byte byte byte" to the tune of the finale of Rossini's William Tell Overture.

While I am sure that the merits of Cutler's stated "low opinion" could be debated, I'm interested to better understand exactly what he was referring to here. There isn't any citation, and I haven't found a good explanation critiquing his criticism.

What specific or fundamental difference between Unix and VMS or Windows NT (which borrowed heavily from VMS) was Cutler pillorying?

Answer 1

The I/O model on "Cutler systems" – RSX-11M, VAX/VMS, Windows NT – is an asynchronous packet-driven I/O model, rather than the fundamentally synchronous I/O model of Unix. At its core, you fire off an I/O request, and get a notification of when it's complete. Meanwhile, execution continues.

Of course, it's trivial for the system to provide synchronous I/O routines on top of that: have the I/O request completion set a condition ('event flag') and then wait on that condition.

In contrast, adding asynchrony on top of a synchronous I/O system is rather more involved.

This, I think, is the nub of Cutler's argument against Unix I/O.

I can provide no citation, however, and I am also biased towards packet-driven I/O, so maybe it's just one (more) man's opinion.

(No implication intended that packet I/O was a Cutler invention – it was certainly present in prior systems, and it was required to implement I/O that way in RSX-11M for compatibility.)

Answer 2

While I am sure that the merits of Cutler's stated "low opinion" could be debated, I'm interested to better understand exactly what he was referring to here. There's no citation, and I haven't found a good explanation critiquing his criticism.

Honestly, at face value, it's a naive criticism. Cutler was not naive, so, it's likely just a sound bite poke at UNIXs development model.

But that said, when you look at the larger picture, notably the "tools" notion of UNIX (as expressed in books such as Software Tools by Kernighan and Plauger) they demonstrate writing code and using things such as getc or getchar along with putc and putchar. The simplistic interface of reading from standard input, processing it, and writing to standard output. It's the essence of the composability of software that "does one thing and does it well".

These are, indeed, "get a byte, put a byte", and are well known to have a higher over head than block reads and writes. If you want I/O to perform well, it's best done in device size blocks (e.g. 1K, or 4K, depends on the device).

Now, in reality, underneath those getc calls are block I/O calls. Underlying code loads up a buffer, and then works through that buffer, refilling as necessary.

There is higher function overhead, but not necessarily I/O overhead.

Perhaps these were naively implemented back in the day (I doubt it).

There is/was nothing stopping UNIX processes from doing more efficient block I/O, but idiomatically, it was not presented that way.

VMS is in many ways the antithesis of UNIX, with its special file formats and dedicated utilities in contrast to the UNIX universal stream of bytes and composable tools model.

Answer 3

At the operating system level – as seen by applications – files in VMS are very record oriented. Guide to OpenVMS File Applications (336 page, 2MB PDF) probably goes into far more detail than anyone should be expected to know, but you can get a feel from the Introduction (emphasis mine):

1.1 File Concepts

A computer file is an organized collection of data stored on a mass storage volume and processed by a central processing unit (CPU). Data files are organized to accommodate the processing of data within the file by an application program. The basic unit of electronic data processing is the record. A record is a collection of related data that the application program processes as a functional entity. For example, all the information about an employee, such as name, street address, city, and state, constitutes a personnel record. Records are made up of fields, which are sets of contiguous bytes. For example, a person’s name or address might be a field. A byte is a group of binary digits (bits) that are used to represent a single character. You can also think of a field or an item as a group of bytes in a record that are related in some way.

VMS normally imposes a very database-like view on files at the operating-system level. In contrast, to Unix a file is just "a bag of bytes" and any higher-level organisation is imposed on it by the application program.

It is possible to create VMS files that are little more than "bags of bytes" (using the "stream" record-format), and – from my experience several years ago – doing so is pretty much mandatory if you want to port a "typical Unix program" to VMS... trying to do byte-by-byte file operations on a file organised (by the operating system) as fixed-length records was a recipe for problems.

VMS's record-level view of files goes quite a lot further than "just" reading/writing in fixed-sized chunks and includes built-in handling of record-locking, indexing, control over how files grow and a whole lot more (over 300+ pages more!) that I never had to get into.

Although I cannot be certain, I strongly suspect it is the lack of this OS-level support for "organised data" files that Dave Cutler was complaining about.

Answer 4

At that time, computing model was very rich API, complex CPUs, complex tools, etc. Filesystem were almost structured files oriented, etc. UNIX came with its "uniform" vision of I/O, everything is a file, a file is just a stream of bytes. That wasn't so easy for people trained on former OSes to understand why UNIX is a good model. While I loosely remember the debate over OSes (I used many of them so I saw how people debated on this), I can remember the same debate over CISC/RISC architectures. The same for compiler, there was a time were people thought that compiler as to be simple translator of very rich programming languages, but now it is more the converse and nobody would try to beat a compiler writing assembler by hand. For I/O I remember all those system engineers and developers which were semi-gods at choosing the right setting to obtain a very tricky effect... I think there is nothing very profound in Cutler's remark, two worlds were in "fight", that's all, and the one that wins in the end is not Cutler's one and he probably knew it).

Answer 5

Having never written Unix I/O code beyond munging text files, read what follows with a grain of salt.

In uni we were a VMS shop. I still recall when the VAX 11/780 in the walled-off computer room was replaced by a single DECStation 3100 on a desk. That machine ran both OSes and we (the OS course's 12-or-so members) began to compare and contrast the approaches while writing our own OS as our 4th year project.

The prof noted that Unix's I/O model relied on the underlying OS to efficiently context switch (which it did), so this was the basis for entire software stack. Even at the application level, the assumption was just "ask for the precise bit of data you want" and it would magically appear. That you might have blocked while it arrived was both invisible and unavoidable.

That model worked well in memory-limited systems, like the PDP-11/20 with a maximum of 32kwords. In this case you had limited space to play with buffers, and I/O was simple and blocking anyway due to the nature of the bus. So to actually transfer that data you needed to context switch anyway, so just do that, bake it into the entier system. In this respect, Unix may be considered well-tuned for minicomputer systems of the era.

But the same is not true for mainframes, which invariably featured some level of autonomous DMA and dedicated buffers. The data is likely arriving rapidly in larger chunks, say 2k HD blocks which magically appear all at once. In this case, the proper model is to process as much of that data as you can before forcing another context switch, because reading one line of text from a 2k buffer and then waiting for the next switch is helping no one.

I vaguely recall a text file floating around usenet comparing the two approaches for a simple text munger running under VMS and Unix on the same machine (a 11/780 IIRC). Even for very simple tasks like those of the unix "tools" pattern, both the application itself and the overall OS performance (total throughput of all running apps) was significantly higher under VMS. Almost all of this was because of avoided switches. I'll see if I can dig it up.

Answer 6

Solely based on the source link to Dave's comment, I can't really tell what he meant.

That said, Benno Rice recently gave an excellent talk at linux.conf.au called What UNIX Cost Us.

The TL,DR of that talk is that many of UNIX's core assumptions, though extensible, no longer apply in current and next gen environments, which makes supporting new paradigms with them extremely difficult.

As an example of this, consider services management. Windows has Services.msc, which was designed to be easy enough for your average high school grad to understand and use. Systemd is a step in that direction for Linux, but is reviled by many for UNIX "nonconformance." Then you have OSes like FreeBSD that supposedly do things the UNIX way, but init is so clunky even FreeBSD's own documentation suggests simply using cron jobs with @reboot instead:

In some cases, it may make more sense to use cron(8) to start system services. This approach has a number of advantages as cron(8) runs these processes as the owner of the crontab(5). This allows regular users to start and maintain their own applications.

The @reboot feature of cron(8), may be used in place of the time specification. This causes the job to run when cron(8) is started, normally during system initialization.