When Intel introduced the 80486 in 1989, they included their first on-chip cache, ostensibly to compete better with Motorola, who had been including on-chip caches for 5 years (MC68020, 1984).
Unlike the Motorola CPUs, Intel went with unified L1 cache for the '486. Then, they switched to separate Instruction & Data L1 cache with the Pentium and its successors. Why the switch, and why not use separate cache for the 80486, if it was obviously better?
I’m not sure the separate cache was “obviously better” back when the Intel designers were working on the 80486, at least, not to the designers in question.
But “better” might not even have been much of a factor. The design history of the cache systems in Motorola and Intel CPUs is quite different, which can explain the different approaches used in the 68040 and 80486.
On the Motorola side of things, the 68010 introduced a loop mode for tight loops, allowing them to run without instruction fetches from memory. The 68020 replaced this with a dedicated 256-byte instruction cache, but no data cache. The 68030 added a 256-byte data cache alongside this, with a modified Harvard architecture. The 68040 revamped the caches and expanded them to 4KiB each. The split cache is a natural progression given the architecture’s history.
On the Intel side of things, caches (apart from the TLB and prefetch buffer) didn’t appear on-CPU until the 486. On the 386, it was common enough to have external caches via the 82385 cache controller; this was unsurprisingly a unified instruction and data cache. So pulling the cache on-die with a single MMU and a unified cache feels logical given the history here.
Both the 68040 and the 80486 benefited from process changes which allowed them to have million-transistor budgets (in both cases, up from under 300,000 transistors in the previous generation to around 1.2 million transistors). The bump seems huge, but once you pull in the FPU, MMU(s), add cache etc., there are still compromises to be made (famously, to the FPU on the 040). In both cases too, the new CPUs were evolutions in their respective architectures, not the huge revamps that the Pentium ended up being; so it’s plausible to imagine the designers working mostly on merging existing external features and improving the core design incrementally, rather than looking around and re-evaluating all the design choices.
By the time they started working on the Pentium, the Intel designers had some feedback from the 486 cache. The 486’s architecture had been “good enough” that they could double and even triple its internal clockspeed; this means that the cache subsystem was sufficient to keep the instruction units busy in many workloads (since clock multiplying without improving the memory bus significantly relies on cache even more heavily). However the 486 did suffer from some level of cache contention, and this became particularly troublesome with the additional processing bandwidth in the Pentium instruction units, especially given the availability of two instruction pipes (the U and V pipes). According to Inside the Pentium (Bob Ryan, Byte Magazine, May 1993), this is the main factor which drove Intel to split its cache into two. In addition, the TLB and cache tags were changed to be triple ported, to allow simultaneous access from both ALUs. The Pentium die layout certainly benefited from the split caches, and this is a common reason to favour split caches (the data and instruction caches are used by different sections of the CPU, and they can be laid out close by).
Importantly given the development practices at the time on x86, the split caches are coherent on x86, so that the caches don’t cause problems with self-modifying code.
cpuid except it didn't exist until late 486, so there weren't any good user-space choices). In practice CPU vendors wanted to not break existing self-modifying code (primitive JITs or whatever). Coherent L1i + pipeline is harder!
Commented
Jun 8, 2019 at 20:15
Stephens Answer already carries most implications, so this is merely an add-on.
First to keep in mind is that the 68k was way more in need of a cache than x86 CPUs, as its memory access was in line with execution, while the x86 prefetch buffer used 'free' cycles to read ahead, thus utilizing the memory much better than the 68k could do (*1).
Next, it depends on your processor structure to make separate caches worthwhile. Both CPUs use a single address space for instruction and data (von Neumann style) thus the simplest way (*2) to speed up memory access (*3) is to add a cache which operates on the plain memory interface without any change to the CPU. Caching that way is a feature of the memory system transparent to the CPU.
Regions that get accessed get cached into faster memory (*4) in hope of future reuse from there. That's what happened with the various cache designs for 286/386 systems. Simple and straightforward.
Now while a simple (unified; *5) cache is a quick and great solution, not all memory locations are equal. Most notably, instructions get reused more often than pure data. So if chip space is scarce (*6) and only a few bytes of cache are possible, then it will be better to reserve them for (hopefully) repeated instructions. That's the way Motorola went on the 68020 by adding 256 bytes of instruction cache.
Such a 'just instructions' cache isn't more complicated than a generic cache. The single difference is that data read access doesn't get cached (*7), everything else works the same. So far the 68020 did not bring a separate cache, but a partial one.
It was the 68030 that introduced a data cache of 256 bytes as well - plus adding a burst mode to read up to 16 bytes at once. Adding a data cache adds complications. Separating instruction and data cache is often described as Modified Harvard. While it's quite easy to add two separate caches on a true Harvard architecture, it's a nightmare for von Neumann style memories. Now each of the caches can hold the same data, so they must be synchronized with each other and memory as well.
Motorola's decision to go with separate caches in the '030 is more likely a result of keeping investment down. After all, the '030 is mainly a shrink of the '020, so adding a data cache (*8) on top might have been less work than redesigning the whole cache system. Also, since the added cache wasn't exactly large, keeping it divided it preserved the speed characteristics the 68020 already showed from its instruction cache.
When the 68030 became available, desktop 80386 systems were already sold with up to 64 KiB of cache. The sheer size of a more than 100 times bigger cache made Motorola's effort seem useless in comparison (*9). Intel kept the single cache for the 80486 (1989) but now integrated 8 KiB of cache right onto the CPU - which didn't stop mainboard manufacturers from adding external caches, now in the region of 256 KiB to 1 MiB.
Fact: Size does matter (and simply beats strategy).
As Stephen already mentioned, these designs were still good to feed the processing units of a 486 fast enough. So when the 68040 came around, Motorola just increased the size to 4+4 KiB cache with just little improvements.
While Motorola's strategy wasn't bad, it wasn't superior either. And Intel's switch to separate the P5's cache into two wasn't driven by some inherent advantage, but by 'need for speed'.
The 486es cache was 4-way associative, meaning each memory location could be buffered in any of 4 cache locations. To find the right location, the tags need to be compared; this takes time. by splitting the cache into two separate parts, this could be reduced to a 2-way system while keeping (in most cases) the same performance - but due to the less complicated hardware, its response time could be improved.
The basic idea of Intel's switch is not so much about specializing on instruction or data, but split the cache between two memory regions that (hopefully) overlap as little as possible, resulting in the same (lower) thrashing rate of a 4-way system but using the faster hardware of a 2-way system. Gaining a higher hit rate within instructions is a rather welcome side effect.
*1 - Motorola tried to counter this with a tiny cache in the 68010 realizing the so called loop mode.
*2 - After adding ever faster RAM.
*3 - Keep in mind, it only makes sense if the processor can handle instruction and data faster than it can be accessed from standard memory.
*4 - The stuff we couldn't afford for all of the RAM.
*5 - Isn't it strange to call something unified if it wasn't divided before? A great example of a retroactive definition we take as equal to its counterpart, despite the fact that it was only coined afterwards.
*6 - And it always is - just the magnitude changes.
*7 - Data write will still be used to update/trash entries, thus allowing to survive self modifying code or data and code intermingled.
*8 - And the MMU.
*9 - No doubt, it was a great benefit compared with the '020, still...
One problem with a split cache is that it will require either that the code cache include sufficient logic to guarantee that if code writes to a region of memory and then jumps there, the CPU will not try to execute stale code from the cache, or else that any code which writes some storage and then jumps there perform an intervening operation to invalidate any involved code-cache entries. The latter approach is much cheaper when practical, but I think Intel recognized it as impossible at least in the short term given people's desire to run existing programs which did such jumps without any cache-invalidating operations.
cpuid which didn't exist yet, or some privileged instructions) between a store instruction and jumping to that address and having code-fetch see it. But in practice x86 vendors have always gone beyond the paper specs to avoid breaking existing code. AFAIK the most any real x86 CPU has ever required is a jump (which restarts code-fetch and thus is sufficient without split caches or branch prediction.) See Observing stale instruction fetching on x86 with self-modifying code answer: you can't.
Commented
Jun 8, 2019 at 20:39
isync or ARM dsb. This lets them cheaply implement split caches (natural for a pipelined design to avoid needing multi-ported cache for fetch + load/store). I'm not sure what m68k required.
Commented
Jun 8, 2019 at 20:42