I guess the difference between "an average programmer" and a compiler is that the compiler has "mechanical sympathy" with the hardware it's compiled to. Also feel the need to quote Donald Knuth / Hoare / Dijkstra, depending on who you ask: "premature optimisation is the root of all evil".
In today's world of cloud computing, it all gets fuzzy: virtual machines, containers and runtime virtual machines (eg Java's Virtual Machine) can all co-exist together. Therefore, compiler micro-optimisations are meaningless in the grander scheme of things - code optimised for a container might be irrelevant on the VM / Physical hardware it runs on.
Of course, if we're talking about bare-metal control, then it matters. However these scenarios are quite niche, unless we're talking about running code on Micro Controllers, then optimising power by optimising CPU cycles is good. x number of CPU cycles costs microamps of battery life, so this could be critical for some applications.

I guess the difference between "an average programmer" and a compiler is that the compiler has "mechanical sympathy" with the hardware it's compiled to. Also feel the need to quote Donald Knuth / Hoare / Dijkstra, depending on who you ask: "premature optimisation is the root of all evil".
In today's world of cloud computing, it all gets fuzzy: virtual machines, containers and runtime virtual machines (eg Java's Virtual Machine) can all co-exist together. Therefore, compiler micro-optimisations are meaningless in the grander scheme of things - code optimised for a container might be irrelevant on the VM / Physical hardware it runs on.

Of course, if we're talking about bare-metal control, then it matters. However these scenarios are quite niche, unless we're talking about running code on Micro Controllers, then optimising power by optimising CPU cycles is good. x number of CPU cycles costs microamps of battery life, so this could be critical for some applications.

Processors have branch condition caches, L1 and L2 caches for RAM to speed up memory access and branching, as well as disk/ssd-backed virtual memory. Processors can also pipeline instructions and effectively run some parts of code in parallel if there are groups of unrelated instructions which are unaffected by the order of which they are executed. Intel did this with their Hyper Threading technology, and there were probably others before them, but I'm not certain who they are without some proper research.

The JVM has a Hotspot compiler. The hotspot compiler converts frequently interpreted portions of byte-code into native machine code to save repeatedly parsing/translating the byte code continuously. Compilers have optimisation too, like in-lining code to save on some kind of machine-code call instruction (which normally might involve additional cycles for saving the return address at the very least). From a heuristics perspective, it's the empirical data which handles on-the-fly optimisation, so they're going to catch stuff you never even thought about.

Much of this depends on the processor, the language, the operating system, the compiler and the data that needs to be processed.