And yet another point on this topic... the "four times" may have been referring to specific, common, platforms.
Older languages generally had global scope and limited subroutine functionality. For instance, FORTRAN had user-controlled scoping and in many cases, there was no local data in a routine. Programs also generally used subs as defined functions, as opposed to a way of organizing code (well...).
In contrast, Algol-derived languages use the block as the primary code organization concept, and programs are generally a collection of subroutine calls. Because the blocks have local scope, every one of these calls generally results in the creation (and destruction) of an activation record. As a result, there is significant overhead in the call dynamics in C (et all) that older languages didn't have.
This led to the widespread use of intermediate systems programming languages, like BLISS mentioned above. On micros, these languages generally combined block-like layout with non-recursive call semantics that didn't require activation records. For instance, Action! on the Atari was generally considered to be about half the speed of hand-coded assembler, whereas C programs were much slower.
While larger platforms like the PDP's and VAXen had larger stacks and userspace controls that aided block-oriented languages, along with the room needed for more optimizations, I suspect at least some of that "four times" was a result of this same effect. Assembler on the same platform could tightly control the calls and unwind with ease, things that did not come to compilers until later.
You can also see this in the performance of systems that were designed to support block-oriented languages; things like the CRISP, the original RISC II and various stack-oriented machines generally offered performance from C that was highly competitive with assembler - that was their entire raison d'etre.