I remember the computer I played Commander Keen on as a child had a turbo button that I was forbidden to touch, what did this button actually do?
The Turbo button originally adjusted the clock speed of the computer between the full speed of the machine and a slower speed intended to be compatible with something more industry standard. It wasn't at all uncommon for software to be written with a specific speed of hardware in mind and either fail to operate completely or operate in a way that was unusable at a faster speed.
(One example of this was the GATO submarine simulator, which was built to run on a stock IBM PC. The faster your computer, the faster your virtual submarine would go. By the 90's, it was easy to find hardware where you could zip across entire virtual world in a matter of seconds. The Turbo Pascal runtime had a more serious issue where programs built using it would fail to run entirely on machines fast enough to cause an overflow during a timing loop on startup.)
PC Accelerator boards had some interesting variations on the theme. Many of these boards could be disabled entirely with a switch to get back to the original, un-accelerated performance of the machine. (This feature wasn't completely without risk, because I remember at least one or two instances where I saw PC's that had these boards installed, but never enabled.) There were also a few boards that had finely adjustable clock rates. Much like overclocking today, the idea was to pick the fastest speed at which your specific hardware was stable. Some of these would also automatically defer selecting the faster clock rate at startup until the BIOS self check had passed. In an effort to avoid this sort of acceleration, IBM put a timing loop in the self check that would fail if the machine detected a faster clock rate... so waiting to select the faster rate would let the self check pass and then accelerate the machine.
All that said, the need for a button to slow a machine down wasn't all that commonplace. At the end of the era of Turbo Buttons, I think the biggest reason to have one was to put the text Turbo on the front of the machine, with whatever connotations of speed it implied.
The turbo button was implemented on machines with CPUs faster than the original IBM PCs. Some software relied on the CPU running at a certain speed to work properly, rather than using some external timekeeping mechanism to avoid running to fast.
With the turbo mode disabled via the button the computer would run at approximately the speed of an original IBM PC, for compatibility with that software. With it enabled the computer would run as fast as it could.
In addition to the other answers given:
I remember newer computers (up to the Pentium class) having such buttons.
I doubt that the Turbo button was connected at all in most later computers.
I suppose that the manufacturers of PC housings did not want change their production and therefore continued producing housings with a Turbo button (which was used for 8 MHz 8086 computers) although the button was not used for newer computers any longer (and Pentium computers could not be switched to 4.77 MHz frequency).
I remember some computers that could be switched between 33 and 66 MHz using the Turbo button. Maybe the manufacturers produced the housings to be compatible with such mainboards (but in 99% of all cases the button was not connected).
On many PC cases, the turbo button would close a contact between the two wires in a twisted pair that connected to it; the far end of the pair would have push-on connectors (sometimes a two-pin 0.1" connector, and sometimes two individual single-pin connectors). If a mother board had a speed-control jumper, the turbo button would often be wired to that, but it could be (and on some systems was) wired in series with speaker, so as to allow an easy "mute" function. Interestingly, some cases had an LED numeric display on the front and used a double-pole switch for the turbo button. Each segment of the LED display could be set using jumpers to be always on, always on, only on if the turbo button was pushed, or only on if it wasn't pushed. Although the button might cause a numeric display to change from e.g. "33" to "16", such behavior wouldn't imply that the button actually caused anything to run faster or slower (or was even wired to anything that would have any effect whatsoever beyond changing the digits).
The turbo button altered the speed on the computer, as some games would reply on the CPU speed in order to determine on how fast the game would run, and this would cause the program to run faster than intended the newer the computer got (however this isn't a problem with today's computers as we can emulate another cpu), so the turbo button was made in order to slow to computer so that it could run these programs. However, some made the computer run faster.