Why was preemptive multitasking so slow in coming to consumer OS's?

Question

Preemptive (rather than cooperative) multitasking was a highly-touted feature for PC's in 1996, with its inclusion in Windows 95 for the first time. It was also highly-touted for 2001 Macs when included with OS X. Earlier versions of both these mainstream consumer OS's supported cooperative multitasking.

Meanwhile, preemptive multitasking was the default for more sophisticated OS's, like Unix and OS/2, even decades earlier. And the Amiga shipped with a consumer OS with preemptive multitasking in 1985.

Notwithstanding the Amiga being 15 years ahead, why was it such a long time for preemptive multitasking to be supported by the mainstream consumer OS's?

Answer 1

Speaking for the Macintosh here.

TL;DR: It wasn't possible to do this in a compatible manner after the hardware was capable enough. Compatible to the already existing application base.

You'll need to see this in context with the heritage of the Macintosh Operating System itself. It was built to run in an considerably limited environment, regarding CPU power and available real memory. Add the fact that the initial release was certainly not production quality code, time pressure was high to get the whole thing running good enough for the highly anticipated release. See Folklore.org for details.

(Lacking memory remapping support in the CPU plus initial floppy-only machines prevented to implement paging in a bearable manner. See Memory Management Unit in Wikipedia for details.)

When RAM got cheaper over time but floppies still were current as permanent storage medium, the Switcher was born by chance. Applications were adapted to better behave with Switcher and vice versa.

From there, MultiFinder was the next step, coming out in 1987. It (also) takes advantage of the fact that there's relatively little to change in Applications for not only sitting in RAM, waiting to be switched to foreground but also being able to run in background. MultiFinder simply exploited the main loop every program was executing, mostly waiting for the user to do something. Applications mostly were stuck in a system routine which delivers an event (Keyboard entry, mouse click, …) to the application program to handle. MultiFinder kind of steals control from applications stuck in this routine and passes control to the next eligible Application. If an Application rarely calls this getNextEvent() routine, it behaves poorly in MultiFinder. Regardless if background or foreground. Best negative example is using Background Printing. Print Monitor searches the Network for the chosen (selected) printer. If the printer is switched off, the whole machine hangs for nearly a minute until Print Monitor returns with an error after a timeout, displaying error condition indicators and calling getNextEvent() so it can handle the user's input.

Over time, developers made Applications more multitasking-friendly by adding getNextEvent() in internal processing routines, so lengthy processing didn't block other applications. This usually doesn't feel as fluent as cooperative multitasking in a graphical environment, though.

(Novell Netware 3 and later also supported cooperative multitasking between extensions of the Kernel, server.exe. These NLMs were loaded at runtime and provided hardware services like drivers for mass storage adapters and network adapters as well as additional network protocols like AppleTalk and TCP/IP, and application support such as pserver.nlm, the print server or database support from Oracle, … I never programmed NLMs but Novell's implementation of cooperation was either very sophisticated and/or in a text based environment, the possibly non-monotonic time slices weren't as apparent as in a GUI environment like the Macintosh ones.)

System 7 added some more code to smooth out multitasking further. I can't prove it but I think, Apple changed the way to decide which Applications were eligible to get CPU control. Kind of dynamic scheduling instead of a dumb cycle through all open Applications: Foreground Application gets control more often than background ones.

In the meantime there were a lot of applications for the Mac. Commercial, free- and shareware. Apple's struggle to keep up with competition while trying to reimplement a new OS with features we today take for granted while keeping compatibility with this pool of Applications can be searched in online media archives of the late 1990s. Search terms include Taligent, Copeland, BeOS and finally NExTStep reborn as Mac OS X.

Switcher was a quick hack; MultiFinder extended the possibilities and that's about it. I think, Apple was too busy creating the next big thing instead of trying to use bolts and nuts to add stuff which involves changing basic OS stuff in an possibly incompatible way.

Btw., the Amiga had the advantage of it's coprocessors relieving the main CPU from certain tasks. I can't prove it but I think that in such an environment, preemptive Multitasking was a better way to exploit this seemingly-parallel execution environment.

Answer 2

Memory protection.

It's not that preemptive multi-tasking is expensive, or hard. It's not. It's easy. It costs (or can cost) essentially the same as cooperative multitasking. You have to save process state in both cases.

But what was holding back the older systems was their early reliance on systems without inherent memory protection, and those legacies lasted long past the availability of hardware that actually had supported Memory Management Units (MMUs).

Take the Mac, for example. Its legacy was the 68000, which did not have direct MMU support.

It was not long before the Macs started coming out with 68020 CPUs (which did have such support). But the OS has to run not just on the new hardware, but also on the old. The two systems are quite incompatible. Plus the actual impact on software design on those systems.

When you start from scratch (like OS/2) then, yes, it's easier.

MacOS heritage held it back for a long, long time before they were able to replace it with with MacOS X via it's Carbon compatibility layer for the software in the new, Unix-ish environment.

The OS for the Apple IIGS was actually a "better" MacOS in some ways, notably in process management and memory management. MacOS 2.0.

Windows suffered similarly. Recall early Windows ran on the 8088. Not the 80286, the 8088. That legacy also burdened it for quite some time.

Code has momentum, choices have consequences. Be amazed they worked at all.

Answer 3

To start with, the only hardware needed for preemptive multitasking is an interrupt capable timer. Everything else can be done in software. Though, some memory management would be helpful. Besides custom solutions, that hardware was already ready available off the shelf for 8-bit CPUs. Beside more generic solutions like TI's 74610 series, more advanced solutions like Motorola's 6829 were available.

Notwithstanding the Amiga being 15 years ahead,

Let's skip that subjective part, OK :))

why was it such a long time for preemptive multitasking to be supported by the mainstream consumer OS's?

Now, hold the horses. I hope you agree that CP/M was a mainstream consumer OS, wouldn't you? DR already offered its preemptive multitasking brother MP/M in 1979. It would work with 'only' 32 KiB, but of course considerably better with several 64 KiB address spaces, one for each process.

This 'huge' memory requirement also marks the primary reason why usage was limited: the price for such a computer. MP/M was targeted at 'power users' with an urgent need to have multiple applications running in parallel. While RAM prices did drop, this need for a use case stayed.

MP/M-86 was available right with the IBM-PC and turned later on into Concurent-DOS. Still it was the missing use case that stopped a broader usage. Not least due to lacking software that would benefit from multitasking at all. Other more limited products like DoubleDOS and Desq had more success in the general public by offering the chance to hold more than one program in memory, handy to reduce load times. Otherwise there was little gain, as these programs worked virtually separated.

More problem-centric solutions like Sidekick did gain much popularity, as they offered at least some (workflow) integration. Apple, on the other hand, did focus from the beginning on a more user centric approach. Already the original Finder supported clipboard exchange and accessories in addition to a (single) main application. With System 5's Multifinder more than one application could be loaded. Still, it took some time until this really got a foothold through better integration.

Bottom Line: It's about the application, stupid.

From a user's perspective there is no real difference between cooperative and preemptive multiprocessing. It's all about what can be done. No matter how we engineering orientated people value these concepts, Multiprogramming, Multitasking, Multiprocessing and all the other Multi* are only tools to enable the user to do his tasks and no real value in themselves. Separated address spaces, preemptive multitasking and so on only became standard when supporting hardware came for free. And at first it only supported programmers in creating applications (or help handling their inability to do so), not offering any direct benefit to the user.

Answer 4

If you look at early (small) computer operating system kernels like CP/M or MS-DOS, there actually isn't that much functionality supplied by those systems. Apart from a thin layer of drivers for keyboard and console, the main function supplied to the applications is the file system driver. CP/M and early MS-DOS didn't even have support for memory management. In an exaggerated sense, both CP/M and MS-DOS weren't much of a "real" OS, but rather very small pieces of code, thrown-together from small drivers and some utility programs.

An OS supporting preemptive multitasking needs to supply many more functions, memory management, interprocess communication, support to allow multiple processes to use the screen concurrently - A multitasking system can't allow programs to access the hardware directly and must instead block other tasks when one of them accesses hardware. So, it is simply a lot more effort to build such a system.

The first home computer to support preemptive multitasking was actually the Sinclair QL in 1984 - It implemented all the above and more in its OS QDOS, and its BASIC interpreter was just one of many possible jobs that could run concurrently. All of this was implemented in only 48 kBytes of ROM on this computer.

Computers like the Macintosh or Atari ST whose operating systems weren't built with multitasking in mind and had a considerable user base and applications suite were much harder to move to a preemptive multitasking system - compatibility with existing application was making the move extremely difficult - MacOS and Atari ST systems that, adopted preemptive multitasking in add-on developments like MultiTOS had large compatibility problems in the beginning and needed to take a lot of compromises to keep compatibility with existing applications (and, thus were obviously much more complicated to develop). The same is true for later PC-DOS multitasking systems like DesqView.

Another reason might have been: It was hard to convince the normal non-power user what they could probably need preemptive multitasking for. The average user argued "I can only work with one program at a time, so what?", and OS concepts that really created a tangible benefit for the end user from multitasking (preemptive or not) were only slowly evolving. Also, keep in mind, the direct benefit an end user has from a preemptive multitasking system over a cooperative one are zero. It's the software developer who gets all the benefits, and after all it is normally the former, not the latter who decides what computer is bought. Thus, as an engineer, it was presumably very hard to convince your marketing department you wanted to build such a system.

Answer 5

The "classic" Macintosh OS, as well as a lot of application software, made heavy use of what Inside Macintosh refers to as "Handles". Instead of keeping a pointer to a region of memory, application software would keep a pointer to a Master Record, whose first item was a pointer to a region of memory. If code was e.g. using a region of memory to hold a 32-bit length, followed by a sequence of short values and wanted to e.g. compute the total of all such values, it could do something like:

long addHandleContents(long **h)
{
  long n = **h;
  long total=0;
  short *p = (short*)((*h)+1);
  while(n--)
    total += *p++;        
}

Any time code performed any system call which might need to allocate memory, it would have to allow for the possibility that the storage associated with any handle that wasn't locked might get relocated. Typically, code would accommodate that possibility by either locking the handles before making such system calls, or by re-fetching the addresses of handles' memory blocks after making such calls. In cases where there were no such system calls (e.g. the code above), being able to access the storage associated with handles directly made things more efficient than would otherwise have been possible.

Some systems avoid such issues by allocating blocks of storage whose address will be fixed throughout their lifetime. Some (like .NET) require that programs contain metadata which would allow the memory manager to, at any time, find all the all the references that exist anywhere in the universe to any unpinned object that will ever be accessed. The Macintosh approach allowed better memory utilization than fixed-address allocators, and didn't need all the extra metadata required of .NET, but requires that programs know when the system might need to relocate storage. While limited preemptive multitasking might have been possible even under classic MacOS, any operation which would need to relocate any handle associated with an application would have to block until that application reaches a "safe state", thus forfeiting much of the benefit of pre-emptive multitasking.

Answer 6

In the case of the Macintosh, Apple kept promising preemptive multitasking in the "next" release, going back as far as version 5 or 6 of the Mac OS, but it never happened until OS X. A version of A/UX (which is preemptive) for later Macintosh systems was released in 1988. Another gripe about Macintosh systems was they didn't have any DMA in the early years, although PC's and even some CP/M systems had it years before the first Macintosh in 1984, and third party vendors made DMA (bus-mastering) SCSI cards for the later Macintoshes long before Apple included that feature in Macs.

For PC compatibles, there was OS/2, a version of which could run on 80286 systems, but many consumers got the impression that OS/2 would only run on IBM's PS/2 systems (and vice versa for some consumers), and not the 386 EISA PC clones.

For Windows, NT 3.5.1 was released in 1993, but NT didn't become popular until NT 4.0 in 1996, and at the time it wasn't as popular as Windows 95 or 98. Windows 98 was followed by Windows ME. Windows NT 4.0 was followed by Windows 2000, XP, Vista, 7, 10 Going back to 1993, Windows 3.11 wasn't preemptive, but did include support for 80386 32 bit flat address mode via win32s extension or winmem32 (only Watcom C/C++ 10.0 compiler supported winmem32 as one of it's memory models at that time).

Answer 7

The PC platform did not get off its DOS compatibility crutch: while the 80286 had segmented protected address modes, they were incompatible with the "real address mode" that the CPU started in and since Intel envisioned real mode only to be interesting for booting, switching to protected mode was not reversible ("triple fault" as a quick way to get back to real mode was more an accidental discovery than a planned feature). Real mode bogged the system down to somewhat more than 1MB of memory (expanded by chipset via EMM memory into what the processor could have addressed perfectly fine in protected mode) and running multiple applications on top of a Window system was not much of a priority given the large constraints.

And the operating system basis, DOS, did not even do cooperative multitasking: pipes, obvious candidates for that, were implemented by writing the output of the first program to disk and then running the second program with input from the temporary file created by the first. This was actually worse than what MP/M (the successor of CP/M, the "inspiration" for DOS) could do.

While there were some multitasking systems built off the protected mode of the 80286 (like Xenix and Eumel/Elan), those were not mainstream.

On the Apple side, Motorola processors did not bother a lot with memory protection. While a separate MMU was supported from the start, it was optional and consequently rarely available except for machines explicitly intended for running UNIX. The 68010 was actually the first CPU that saved enough information at a page fault to restart instructions: previous versions were not suitable for running copy-on-write schemes and similar demand-paging features.

However, the 68012 with a larger physical address space and the 68020 with 32-bit data busses throughout and quite extended addressing modes still did not come with a built-in MMU. It's just with the 68030 that the MMU became an on-chip feature (and with the 68040, the mathematical coprocessor, at least to a good degree). However, either were considered mostly overkill for home computers.

Intel was faster with the 80386 which had a 32-bit mode (the 68000 was basically a "32 bit" mode from the start) and a fully virtualizable processor with a paged MMU. This was so completely overkill with regard to the active market Intel was pitching that I have no idea how the engineers managed to push this approach (rather than integrating a mathematical coprocessor, for example) through into silicon. Uptake was rather tepid at first, but it was what started Linux off the ground after a few consumer-level UNIXes like "Interactive Unix" and the proprietary clone "Coherent" were coming about.

Windows 95 eventually got to use the 80386 modes as well, but it required significant engineering and defining a Windows-3 like OS interface over the 32-bit protected mode. The consumer Windows versions were bogged down by history, and just with Windows XP finally the more capable Windows NT approach replaced the consumer line completely.

Apple had in the mean time successfully relied on cooperative multitasking, relocatable code, relocatable memory allocation and was getting long in the tooth with MacOS. In fact, they were in dire straits, finally breaking free by basing MacOSX on a BSD running atop the Mach microkernel. They managed to pull off this rather audacious move (under Jobs) while Microsoft spent decades getting its operating system struggling through compatibility issues. Later on, Mac also managed switching to different CPUs (first PowerPC, later Intel and ARM) while Windows never managed to crawl away from binary compatibility with DOS systems.

So in a nutshell: preemptive multitasking needed additional hardware for a long time, multitasking at all was not really in the DOS/Windows scheme for a long time, impressive built-in memory protection in consumer-level devices came late with the 80386 and even later with the 68030, and MacOS was rather successful with its cooperative multitasking schemes and almost missed jumping off.

Answer 8

You underestimate the power of existing programs.

At that time everybody was using MS-DOS/PC-DOS on PC's which for all practical purposes was a program loader for running one program at a time, which then was in full control of the whole machine.

For multitasking to be generally interesting you would need to be able to run multiple DOS programs simultaneously, or be restricted to a few select programs. It was not until the 386 was in widespread use this could be done well, and the users could see the utility of staying in Windows to run multiple programs at the same time.

(Until then Windows was typically just something you started to fire up an application, and quit again when you needed to do something else - this changed with Windows 95)

Answer 9

Mostly legacy reasons. Early affordable computers didn't have enough RAM to make multitasking worthwhile, so they used monotasking operating systems like DOS and MacOS.

In 1985 the Amiga was launched with a preemptive multitasking operating system, but it was a brand new system starting from scratch. Lack of business software meant it struggled to gain a footing in offices, it was mostly sold for games and creative uses.

PCs and Macs had a lot of existing software written for their non-multitasking operating systems, and adding preemptive multitasking without breaking those apps was going to be very difficult. Before multitasking many apps accessed hardware directly and didn't even consider things like shared filesystem access, and of course were not event driven.

Microsoft took the path of running older apps in a partially virtualized DOS environment. MacOS introduced cooperative multitasking as an option and spend many years laying the groundwork for preemptive multitasking, so that when the time came the switch over was easier. MacOS also had less commercial software to support, particularly vertical apps, and was able to get most of the big ones updated.

Answer 10

Besides all the technical reasons, personally I think that there was a lot of company policy involved.

In the 1980s, microprocessors like the 68000 and operating systems like Unix or OS-9 were there to build powerful multi-user machines, with the potential to become competitors to the established mainframe world.

What happened then?

IBM, earning lots of money from their mainframe branch, launched a personal computer based on a limited processor with a limited operating system that surely wouldn't support multi-user or pre-emptive multi-tasking. This machine would surely not compete with their mainframe world.

And all the IT world jumped onto this platform, no longer devoting resources to the more advanced processors and OSes. This prolonged the lifetime of the mainframe dinosaur world by at least a decade.

So, I think it was a very clever business decision of IBM to launch such a limited machine like the IBM PC. I never understood how they convinced the software industry to nearly exclusively support this platform and nearly forget about Unix for the next decade.