In the 70's and 80's RAM chips worked at a lower frequency than the CPU. That is, the processor worked at a frequency higher than the RAM.
Not really. At least with microprocessors, RAM was usually equal or faster than the CPU. RAM being slower is a development of the 1990s and later again, with micros. After all, the same cycle has been run through with mainframes.
We have that the CPU is idle while waiting for the next command.
Not really. A CPU waiting for data is a relatively new thing - again for micros.
One of it - make a CPU with microcode. While the CPU is waiting for a command from the RAM, the microcode is running at the CPU frequency (for example, doing a complex command on registers).
Nope. Microcode is not made to 'fill time' but to simplify CPU design (save on components) and/or run more complex operations.
Or you can add cache mem in CPU.
Well, yes, but it doesn't have to be in CPU. After all, Cache is nothing else than faster memory which usually is only affordable in comparably small quantities - otherwise it would be used for all memory.
Is it possible to do so (approximately)?
Only the second and only in part.
What to do?
In the end it's all about storage attachment and storage hierarchy(*1).
On a more detailed level there are four basic methods to improve average RAM access time:
- Using regions of faster RAM for critical data/code.
- Using interleaved memory regions.
- Using a wider memory interface.
- Using a cache.
#1 Faster Regions
The first method was very common at a time when vastly different technologies were used as RAM. A good example is a rather early machine such as Zuse Z23 where main memory was made from drum, but the first 256 words were core. Likewise the (way bigger) Univac 1105 whose base memory had 8 KiWords core plus 16 KiWords of drum (*2).
For program logic there was no difference between either type of memory. It was one continuous address space, but code - and especially data - stored in the first 8 Ki could be accessed without any waiting for the drum to rotate and deliver a random word (*3).
In a way, many modern micro processors support a similar working. They allow locking cache lines to a certain memory address. That is, those lines will be taken out of regular cache operation, loaded once with the desired content and never overwritten by other data, thus making sure that those code/data sections are always ready at maximum speed.
Another more modern example, though for a different reason, was the Commodore Amiga. While all its memory was semiconductor of equal speed, one portion (Fast RAM) could be accessed at full CPU speed, while another (Chip-RAM) was shared with I/O. Thus it was advised to store code and non I/O-related data whenever possible always in Fast RAM.
#2 Interleaved Memory Regions
If a memory device takes a certain time before it can be accessed again, why not use several memory devices, each with its own cool down, independently? Of course this would require an access scheme that (hopefully) allows predictable access patterns. Luckily reading code for execution is (mostly) exactly that: strictly sequential RAM access.
Having two RAM devices (banks) with interleaved addresses will give twice the RAM speed. As long as access is sequential. But even a random access might have a 50% chance for hitting the right (next) bank (*4). Having four quadruples RAM speed, and so on.
Interleaved memory blocks have been used since way back in core times - at least with mainframes where one had to have multiple core blocks anyway. It can be traced back to at least the 1961 IBM 7030, and was a common technique with /360(ish) Mainframes throughout the 60s and later.
It's all common with PCs supporting parallel banks since 286 times (*5).
Another modern analogue is RAID arrays.
#3 A Wider Memory Interface
There is no reason to tie the width of a memory interface to the internal word width of a CPU. A very common example are 8-bit devices on a 16-bit CPU. Only half the CPU width is used to access data. The same can be done by widening - like giving that 16-bit CPU a 32-bit memory interface. So whenever it accesses a 16-bit item from RAM, 32 bits are read at once. One half is directly forwarded to the CPU, the other half is latched and kept in case the following access needs it - which is again true for all sequential access.
Like before, doubling size doubles bandwidth. That method became quite popular with mainframes, by 1980 reaching memory interface widths of up to 1024 or 2048 bit (128/256 bytes) - all for a 32-bit CPU.
For microprocessors the Pentium may qualify as the first mainstream CPU utilizing this. While being a 32-bit CPU like the 486, it featured a 64-bit (8-byte) memory interface to double RAM bandwidth.
The method was (and is) quite popular with GPUs (graphic cards). Their access pattern is even more linear than for a standard CPU, so even the 1999 GForce 256 featured a 128-bit interface. By 2003 Nvidia reached 256 bits (NV42) which is more or less the standard up to today (*6).
Such wide memory interfaces are not only a precursor of cache in the sense that they prepare additional data for future access, but also by enabling fast cache loads. Which brings us to the last item:
In some way a cache is the very same as item #1. A dedicated, faster storage area for most important data. Except now no longer the programmer decides (*7), but the CPU hardware/software does so. This takes some additional hardware, so it took some time for technology to be applied (for a more in depth history see here).
Caches do not obsolete #3..4, but rather extend thereupon.
*1 - Think of the basic hierarchy of
each faster than the previous, but more expensive and smaller in capacity. In the /370 world this is the basic way to look at all storage. It goes in many more stages from cache all the way to tape libraries and even offline storage like card stacks :))
*2 - Fully expanded 16 Ki of Core and 32 Ki of Drum.
*3 - This is not entirely correct, as programs could be written (stored) in a way that data and code were spread out to the right way to eliminate any waiting. Of course, it was less effort if you could avoid needing such an arrangement.
*4 - With careful alignment, like back in the days of drum memory, this can be made 100% again :))
*5 - Most recent are putting it to the extreme by handling those blocks as independent memory channels, but that's a different story.
*6 - Further growth didn't stop for missing performance gains - there are chips with 384 and 512 or multiples of 256 - but number of pins and resulting board design will get exponentially more expensive, so such wide interfaces are only useful for extreme applications where money is of lesser concern.
*7 - Well, he still can, but that's very fine fine-tuning.