Older systems, such as the NES, would often access the ROM in real-time rather than caching the data in system memory, sometimes even reading the same data every single frame. This would result in an immediate and dramatic effect when the connection to the cartridge was broken. Newer systems, such as the Nintendo DS, would instead load from the cartridge into memory* only when needed. Because of this, breaking the connection to the cartridge would result in the system freezing the next time data needed to load (e.g. when a new entity is about to spawn).
My understanding is that the older behavior was common when memory was slow enough that reading from ROM was not significantly slower than reading from memory, and limited enough in size that keeping an entire "scene" in memory was not practical. Only when memory capacities got larger and the RAM became significantly faster than ROM did this change.
What did Game Boy Advance cartridges do? For example, was audio constantly streamed from the cartridge in real-time, or was it cached in memory and played from there? Is there any data that is read from ROM every single frame, rather than being read from memory? In other words, was data constantly being read from ROM, or only when "loading" new areas or entities?
* Interestingly, it seems like the Nintendo DS was unable to execute directly from ROM (it wasn't mapped to an address) and needed to first copy to system memory. I believe this was due to copy protection.