26

When Intel introduced the 80486 in 1989, they included their first on-chip cache, ostensibly to compete better with Motorola, who had been including on-chip caches for 5 years (MC68020, 1984).

Unlike the Motorola CPUs, Intel went with unified L1 cache for the '486. Then, they switched to separate Instruction & Data L1 cache with the Pentium and its successors. Why the switch, and why not use separate cache for the 80486, if it was obviously better?

  • 7
    Not specific to Intel, but stackoverflow.com/questions/55752699/… discusses unified vs split caches. – Eugene Styer Jun 6 at 22:03
  • 3
    A possible consideration could have been maintaining functionality of existing software relying on immediate availability of self-modified code, while giving developers time to adapt. – Leo B. Jun 6 at 23:32
  • @Leo why would a separate instruction cache cause problems for self-modifying code? The x86 caches are coherent. – Stephen Kitt Jun 7 at 5:25
  • 2
    @StephenKitt The decision to make caches coherent had to be made. I/D coherency is costly, and with proper programming discipline it is not necessary. – Leo B. Jun 7 at 15:55
  • 1
    @TobySpeight By "developers" I meant both hw and sw developers :). Whoever adapts first. – Leo B. Jun 7 at 15:58
42

I’m not sure the separate cache was “obviously better” back when the Intel designers were working on the 80486, at least, not to the designers in question.

But “better” might not even have been much of a factor. The design history of the cache systems in Motorola and Intel CPUs is quite different, which can explain the different approaches used in the 68040 and 80486.

On the Motorola side of things, the 68010 introduced a loop mode for tight loops, allowing them to run without instruction fetches from memory. The 68020 replaced this with a dedicated 256-byte instruction cache, but no data cache. The 68030 added a 256-byte data cache alongside this, with a modified Harvard architecture. The 68040 revamped the caches and expanded them to 4KiB each. The split cache is a natural progression given the architecture’s history.

On the Intel side of things, caches (apart from the TLB and prefetch buffer) didn’t appear on-CPU until the 486. On the 386, it was common enough to have external caches via the 82385 cache controller; this was unsurprisingly a unified instruction and data cache. So pulling the cache on-die with a single MMU and a unified cache feels logical given the history here.

Both the 68040 and the 80486 benefited from process changes which allowed them to have million-transistor budgets (in both cases, up from under 300,000 transistors in the previous generation to around 1.2 million transistors). The bump seems huge, but once you pull in the FPU, MMU(s), add cache etc., there are still compromises to be made (famously, to the FPU on the 040). In both cases too, the new CPUs were evolutions in their respective architectures, not the huge revamps that the Pentium ended up being; so it’s plausible to imagine the designers working mostly on merging existing external features and improving the core design incrementally, rather than looking around and re-evaluating all the design choices.

By the time they started working on the Pentium, the Intel designers had some feedback from the 486 cache. The 486’s architecture had been “good enough” that they could double and even triple its internal clockspeed; this means that the cache subsystem was sufficient to keep the instruction units busy in many workloads (since clock multiplying without improving the memory bus significantly relies on cache even more heavily). However the 486 did suffer from some level of cache contention, and this became particularly troublesome with the additional processing bandwidth in the Pentium instruction units, especially given the availability of two instruction pipes (the U and V pipes). According to Inside the Pentium (Bob Ryan, Byte Magazine, May 1993), this is the main factor which drove Intel to split its cache into two. In addition, the TLB and cache tags were changed to be triple ported, to allow simultaneous access from both ALUs. The Pentium die layout certainly benefited from the split caches, and this is a common reason to favour split caches (the data and instruction caches are used by different sections of the CPU, and they can be laid out close by).

Importantly given the development practices at the time on x86, the split caches are coherent on x86, so that the caches don’t cause problems with self-modifying code.

  • 1
    Solid answer. I saw something in my own research suggesting that the physical placement/location on-die of the cache was critical for minimizing cache latency, and this led some designers to Harvard arch. So wondering if that also played a role with Intel switch for P5. – Brian H Jun 7 at 11:18
  • Yes, I came across that too; if (and this is a big if) the on-die layout of the 486 is anything like this block diagram, it wouldn’t be an issue there, but it could easily have been a problem on the Pentium (and also on the 68040). I imagine there are annotated die shots of 486s and Pentiums somewhere... – Stephen Kitt Jun 7 at 11:31
  • 2
    It looks like the Pentium layout benefits from separate caches. – Stephen Kitt Jun 7 at 11:36
  • @BrianH: Split L1 caches are certainly more natural for a pipelined CPU that's normally fetching instructions in parallel with data loads / stores. Two separate caches are cheaper to build than 1 larger multi-ported cache. And yes, tightly integrating the L1d with load/store ports and L1dTLB, and L1i with L1iTLB and instruction fetch, is another well-known advantage of split caches, as @HadiBrais discussed in an SO answer. Many wires (silicon or metal) over long distances are something to avoid. (P5 has 64-bit wide cache access paths). – Peter Cordes Jun 8 at 20:09
  • 2
    @BrianH: Most classic-RISC ISAs (like MIPS that were designed from the ground up for pipelined implementations) do not have coherent instruction caches: you have to run a sync/flush instruction before you can safely jump to an address where the CPU recently used store instructions to store new machine code. x86 on paper required a serializing instruction (like cpuid except it didn't exist until late 486, so there weren't any good user-space choices). In practice CPU vendors wanted to not break existing self-modifying code (primitive JITs or whatever). Coherent L1i + pipeline is harder! – Peter Cordes Jun 8 at 20:15
21

Stephens Answer already carries most implications, so this is merely an add-on.

First to keep in mind is that the 68k was way more in need of a cache than x86 CPUs, as its memory access was in line with execution, while the x86 prefetch buffer used 'free' cycles to read ahead, thus utilizing the memory much better than the 68k could do (*1).

Next, it depends on your processor structure to make separate caches worthwhile. Both CPUs use a single address space for instruction and data (von Neumann style) thus the simplest way (*2) to speed up memory access (*3) is to add a cache which operates on the plain memory interface without any change to the CPU. Caching that way is a feature of the memory system transparent to the CPU.

Regions that get accessed get cached into faster memory (*4) in hope of future reuse from there. That's what happened with the various cache designs for 286/386 systems. Simple and straightforward.

Now while a simple (unified; *5) cache is a quick and great solution, not all memory locations are equal. Most notably, instructions get reused more often than pure data. So if chip space is scarce (*6) and only a few bytes of cache are possible, then it will be better to reserve them for (hopefully) repeated instructions. That's the way Motorola went on the 68020 by adding 256 bytes of instruction cache.

Such a 'just instructions' cache isn't more complicated than a generic cache. The single difference is that data read access doesn't get cached (*7), everything else works the same. So far the 68020 did not bring a separate cache, but a partial one.

It was the 68030 that introduced a data cache of 256 bytes as well - plus adding a burst mode to read up to 16 bytes at once. Adding a data cache adds complications. Separating instruction and data cache is often described as Modified Harvard. While it's quite easy to add two separate caches on a true Harvard architecture, it's a nightmare for von Neumann style memories. Now each of the caches can hold the same data, so they must be synchronized with each other and memory as well.

Motorola's decision to go with separate caches in the '030 is more likely a result of keeping investment down. After all, the '030 is mainly a shrink of the '020, so adding a data cache (*8) on top might have been less work than redesigning the whole cache system. Also, since the added cache wasn't exactly large, keeping it divided it preserved the speed characteristics the 68020 already showed from its instruction cache.

When the 68030 became available, desktop 80386 systems were already sold with up to 64 KiB of cache. The sheer size of a more than 100 times bigger cache made Motorola's effort seem useless in comparison (*9). Intel kept the single cache for the 80486 (1989) but now integrated 8 KiB of cache right onto the CPU - which didn't stop mainboard manufacturers from adding external caches, now in the region of 256 KiB to 1 MiB.

Fact: Size does matter (and simply beats strategy).

As Stephen already mentioned, these designs were still good to feed the processing units of a 486 fast enough. So when the 68040 came around, Motorola just increased the size to 4+4 KiB cache with just little improvements.

While Motorola's strategy wasn't bad, it wasn't superior either. And Intel's switch to separate the P5's cache into two wasn't driven by some inherent advantage, but by 'need for speed'.

The 486es cache was 4-way associative, meaning each memory location could be buffered in any of 4 cache locations. To find the right location, the tags need to be compared; this takes time. by splitting the cache into two separate parts, this could be reduced to a 2-way system while keeping (in most cases) the same performance - but due to the less complicated hardware, its response time could be improved.

The basic idea of Intel's switch is not so much about specializing on instruction or data, but split the cache between two memory regions that (hopefully) overlap as little as possible, resulting in the same (lower) thrashing rate of a 4-way system but using the faster hardware of a 2-way system. Gaining a higher hit rate within instructions is a rather welcome side effect.


*1 - Motorola tried to counter this with a tiny cache in the 68010 realizing the so called loop mode.

*2 - After adding ever faster RAM.

*3 - Keep in mind, it only makes sense if the processor can handle instruction and data faster than it can be accessed from standard memory.

*4 - The stuff we couldn't afford for all of the RAM.

*5 - Isn't it strange to call something unified if it wasn't divided before? A great example of a retroactive definition we take as equal to its counterpart, despite the fact that it was only coined afterwards.

*6 - And it always is - just the magnitude changes.

*7 - Data write will still be used to update/trash entries, thus allowing to survive self modifying code or data and code intermingled.

*8 - And the MMU.

*9 - No doubt, it was a great benefit compared with the '020, still...

  • 3
    Your answers are always fantastic, extremely informative and interesting. But I find your use of footnotes makes the post difficult to read. I have to keep scrolling down the page to read the footnotes, and then I lose my place in the text. I have a similar complaint about footnotes in books, where I have to keep flipping to the end to look them up every few sentences. If there's any way you can incorporate the footnote information directly into the flow of the answer, I think that would be a huge improvement. If it doesn't make sense to just insert directly, then maybe in parentheses? – Cody Gray Jun 8 at 3:43
  • 3
    @CodyGray Well, it's a way to reach two divergent goals at the same time. It may not seam like it, but I try to keep my answers short and focus on the question. Then again my inner nerd shouts out all the additional facts, informations and thing he feels noteworthy. The alternative would be to add it in parentheses. Just, except for some really short ones, they will kill the reading as well (and some are several paragraphs or even lists). So I go with the footnotes to get at least some of the optional information out of the way. – Raffzahn Jun 8 at 8:11
  • For a 4 way associative cache, wouldn't the 4 tags be compared in parallel (4 comparators), similar to "associative or content addressable" memory? Didn't the TLB already do this (parallel compares)? – rcgldr Jun 8 at 14:46
  • "Modified Harvard" to describe split caches fits best on ISAs where I-cache is not coherent, and you do need a barrier instruction before jumping to newly-stored code. This is the case on most RISC ISAs, including ARM, so it removes a major complications for implementing split caches. (And pipelined / OoO exec; modern Intel snoops for stores to instruction addresses that are in flight in the pipeline and does a pipeline nuke after a store within +-2kiB of an instruction in the pipeline, or something like that.) – Peter Cordes Jun 8 at 20:30
  • 2
    @CodyGray: I sometimes put footnotes at the bottom of a section or paragraph so they're not far below the reference. Instead of all the footnotes at the very bottom of my answer. I think that could work here. (I usually use parenthetical asides for things as short as these footnotes, reserving footnotes for things that are a whole sentence. But that does lead to clutter so I see the benefit here.) – Peter Cordes Jun 8 at 20:33
2

One problem with a split cache is that it will require either that the code cache include sufficient logic to guarantee that if code writes to a region of memory and then jumps there, the CPU will not try to execute stale code from the cache, or else that any code which writes some storage and then jumps there perform an intervening operation to invalidate any involved code-cache entries. The latter approach is much cheaper when practical, but I think Intel recognized it as impossible at least in the short term given people's desire to run existing programs which did such jumps without any cache-invalidating operations.

  • 2
    On paper x86 (used to?) require a serializing instruction (like cpuid which didn't exist yet, or some privileged instructions) between a store instruction and jumping to that address and having code-fetch see it. But in practice x86 vendors have always gone beyond the paper specs to avoid breaking existing code. AFAIK the most any real x86 CPU has ever required is a jump (which restarts code-fetch and thus is sufficient without split caches or branch prediction.) See Observing stale instruction fetching on x86 with self-modifying code answer: you can't. – Peter Cordes Jun 8 at 20:39
  • 1
    RISC ISAs were designed later than x86, and most (all?) do not have coherent I-caches; a flush/barrier instruction is required to avoid stale instruction fetch. e.g. PowerPC isync or ARM dsb. This lets them cheaply implement split caches (natural for a pipelined design to avoid needing multi-ported cache for fetch + load/store). I'm not sure what m68k required. – Peter Cordes Jun 8 at 20:42
  • @PeterCordes: On the 8088 and 8086, instructions that altered the PC would forcibly clear the pre-fetch buffer; and I don't remember any other documented means of doing so. Since most code runs in read-only segments, invalidating the code cache when a segment is switched from writable to executable, or any attempt is made to write the current executing segment, would probably not impair performance too badly in most situations. – supercat Jun 9 at 6:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.