In general, no. Cartridges from the first few generations were really only breakouts for ROM chips and thus were mostly a collection of address and data lines with a +5V and GND at minimum and perhaps a few others. In the case of the Intellivision, it also had SYNC, serial lines and a video passthrough (to support IntelliVoice, Atari 2600 module, etc.).
The consoles were designed to look for a cartridge and start executing code on them immediately. If there was no ROM, there was no code to run and thus the machine either sat there getting the same empty signals into the CPU (effectively doing nothing) or if more advanced, the minimal firmware was able to display some kind of message (e.g. ColecoVision error screen or how the NES just blinks, though that behavior is more the 10NES lockout chip).
The risk of damage to a console, as far as I've ever been able to determine, comes from the possibility of static buildup on the cartridge or device being plugged in and having this discharge happen on one of the address or data lines. As such, many "hot swappable" connectors tend to have the pins for the GROUND slightly longer than anything else (e.g. USB does this) so that any discharge is safe before connecting the "fragile" data lines.
That being said, turning on a console without a cartridge should not be a problem since the device will either not work or would have been designed to handle the situation. As far as (un)plugging a cartridge with the power on, it is in general not a good idea to do this if you can avoid it. However, consumer devices like cartridge video game consoles were targeting a non-technical audience and typically were designed to account for people who didn't RTFM and wouldn't have thought that unplugging a cartridge would potentially do damage.