When shopping for a VGA ISA card, how might I know if it would be compatible with an IBM 5162? Do all 8-bit and 16-bit ISA VGA cards work, or only some?
The 5162, more commonly known as the IBM PC XT/286, uses a 16-bit expansion bus. Any 8-bit “ISA” VGA card should work fine (but they are rare), and most 16-bit VGA cards should work too... Ironically, the 8-bit Hercules VGA card is known to cause issues. Some 16-bit cards designed for the PC AT won’t fit in the XT/286’s case and therefore can’t be used either.
Compatibility problems are mainly a concern with older systems, because of their 8-bit-only bus, and/or their 8088 CPU. A few 16-bit VGA cards work in IBM PCs and XTs; the Vintage Computer Forum has a list. Basically, older 16-bit VGA cards made by ATI, Cirrus Logic, Oak Technology, Trident, TSENG, Video Seven, and Western Digital (Paradise) are possibilities; some will work without configuration, others need jumpers to be set appropriately. This page lists 8- and 16-bit cards which are known to work in IBM 5150s.
As Brian Knoblauch points out, plugging a 16-bit card in an 8-bit slot means that part of the edge connector hangs off the end; this is perfectly safe as long as there are no components in the way on the motherboard, and as long as you don’t run into a slot with an edge that’s too thick to fit in between the connectors (it happens sometimes, so don’t force a card in). I couldn’t find an example photo of this in my brief search, but this works in the same way that a PCI-X card can usually be plugged into a PCI slot, with an edge connector hanging off the end (see this article for an example).
If you don't mind soldering and making/ordering custom PCBs, you can actually build your own VGA card. Sergey Malinov has schematics on his website: http://www.malinov.com/Home/sergeys-projects/isa-supervga
It is based around a Trident TVGA9000 chipset, which you can get as New Old Stock off of eBay or similar sites.
The card even has a fix for IBM XTs which do not implement the ALE line correctly.
I don't recall if the 5162 had the same issue as the 5160, but in the latter the 8th slot had some sort of difference in timing, and some cards had an issue with that. If your card doesn't work in slot 8, try it in another slot before giving up.