The way I understand it, ROMS are like virtual games, and emulators are like virtual game consoles, or handhelds. What I don't understand is how there are ROMs for arcade games, which don't have removable cartridges. All arcade machines are different (at least to my knowledge), so wouldn't an arcade ROM have to record the entire arcade machine's hardware? And if it does, what is the emulator for?
The other answers already covered a lot, but there is something else that is important but which hasn't already been addressed in detail:
Despite appearances to the contrary, arcade machines are quite frequently not built with unique hardware.
For example, the hardware originally designed for Galaga was used for several additional games, including well known titles like Dig Dug and Xevious. The system can be changed to play a different game by taking the ROM chips out of its board and putting different ones in.
Starting in the late 90s/early 2000s, many arcade systems were based either on PC hardware or rebadged consoles (e.g. you'll find quite a few arcade machines based on PlayStation hardware from various vintages), so the concept of a ROM for these is much more like a standard console emulator, rather than actually being ROM chips as in the older systems.
So, while in your question you assume that arcade machines don't have removable cartridges, in actual fact at least in many instances there is a concept very similar to the cartridge of a console system -- the only difference is that it isn't exposed to the user, and only really exists to simplify design and production of multiple games that require hardware with similar capabilities. But fundamentally, it works the same way: the majority of the hardware in the machine stays constant, but you can pull a module out of it and plug and new one in, and it will change what game is played by the machine.
The way I understand it, ROMs are like virtual games,
Not really. ROMs are a piece of hardware storing a bit image. Like a disk, a tape or a punch card. It holds an image of the game's software.
and emulators are like virtual game consoles, or handhelds.
What I don't understand is how there are ROMs for arcade games, which don't have removable cartridges.
These machines still have ROMs. Where else should the game's software be stored (*1) and without software, there would be no game.
Your confusion might come from a different use of the term ROM in a gamer (emulation) context compared to real hardware. With emulators the term ROM is used in a somewhat translated and abbreviated way. Instead of saying 'file with the ROM image of game XYZ' people just say 'XYZ ROM'. Within this subculture it is a mostly understandable term, outside it may seed confusion with the basic term ROM describing a piece of hardware, not the image within (copied into a file).
All arcade machines are different (at least to my knowledge),
Yes and no. Some arcade boards could be used for different games - when fitted with new ROMs (for software but also graphics encoding as well (*2)).
so wouldn't an arcade ROM have to record the entire arcade machine's hardware?
A ROM does not record the hardware - nor does it record anything.
And if it does, what is the emulator for?
For running the game stored in the ROM? Someone has to 'do' the hardware.
thinking of it, could it be that your confusion comes from playing various games of different machines on MAME without the need to specify some hardware and have MAME magically running whatever image it gets presented?
MAME is, as the name "Multi Arcade Machine Emulator" implies, an emulator made to emulate many different machines. In its actual implementation it features a system of databases to identify various game images via fingerprinting. It's basically doing some checks/checksums over the image and then looking up if the result is known. If so, the information about what machine and which settings are needed is retrieved from the database, and used to configure the needed emulation.
Using fingerprinting and its database enables MAME to offer zero effort configuration without falling into a copyright trap of delivering the images.
All of this will of course not work with a game image unknown to its database. In this case one can setup several configuration files with all the needed information and run it... except, that's something 99.99% of all users never experience.
Emulator-wise, MAME is a framework offering an API to handle synchronized emulation of specific emulators for hardware (like CPUs, graphic controllers and so on), which are configured to act like the machine in question. If there is new hardware to be emulated, programmers still have to sit down and add a program for each missing piece and a configuration for a whole machine (or several thereof), but no longer build the framework.
After about 20 years of development, the number of components without a fitting emulation is rather small.
*1 - Well, some do have different means, but that's a different story.
*2 - Some even had a cartridge system to make them fast field upgradeable.
A ROM, short for Read-Only Memory, is computer chip with bits stored in it. Unlike RAM chips, the bits are persistent (remain when the power is turned off) and (usually) require a special programming process to change them.
An arcade machine of 80s or 90s vintage is just a computer (often a custom-built, somewhat special-purpose computer) which reads the code for the game from these ROM chips and executes it. The ROM chips may be plugged directly into the system board or plugged into a cartridge that's plugged into the system board. In the latter case these work like 80s and 90s home video games such as the Nintendo NES, but you don't usually see the cartridge because it's locked inside the cabinet.
Since the ROMs just contain bits, these can be stored in a different way; in the emulation community the sequence of bits is often stored in a file. This is called a "ROM image," but some people may just call it a "ROM" for short.
So no, a ROM in this sense is not really a "virtual game" any more than a PlayStation disc is a "virtual game"; a more reasonable definition is that it is the game, or at least the non-hardware part of it.
The hardware, emulated or original, when applied to the code from the ROM, runs the game.
(This explanation has been simplified to better communicate the main point; don't take this as exactly describing all systems.)
The ROM is the software, in all cases. It stands for "Read Only Memory" and it stores the static code and the data needed for the game (i.e. the Software).
The arcade cabinet is the hardware. It's the same as any computer or game console, and so it needs Software (ROM) to do anything useful.
An "emulator" is a piece of software that runs on one piece of hardware (i.e. your PC) and emulates another piece of hardware (old computer, game console, arcade machine, etc.). Without these definitions, I can see where it might be confusing.
The arcade ROM contains only the software that runs on the arcade machine. The hardware part is written into the emulator.
P.S. Arcade machines don't have removable cartridges but they do have removable ROMs. This means arcade motherboards can support multiple games just by replacing the ROM chip(s) (see Jules' answer for examples).
A ROM cartridge is a cartridge that contains ROM chips.
Although arcade machines do not have ROM cartridges, they do have ROM chips. You PC or Laptop also has ROM chips (usually EEPROMs now; electronically erasable & programmable ROM. These are the chips you rewrite when you do a BIOS update).
A ROM doesn't store the entire architecture of the machine it runs on,any more than a software CD doesn't include a copy of your PC. If you try to run it on a different processor than it's made for, the bytes in the ROM mean different instructions, so it won't run. I was going to say "it assumes you have the right CPU & hardware", but it doesn't assume anything. It's just a series of bytes. How those bytes are interpreted (music, images, games) is up to the device.