This question is a bit hard, as 'IBM-Compatibility' isn't as clear defined as advertisement wants to make us believe. In fact, next to any IBM machine could be classified as not compatible, starting with the AT.
Short answer: All PCs are some kind of quantum mechanic device: they are compatible and not compatible at the same time. Only if the system gets observed (aka a program gets started), a definite state emerges. A state different for each program possible.
Compatibility usually falls into two categories: Hardware and software compatibility. Although easy definitions, they often get mixed up by hardware being blamed for software problems.
Hardware compatibility describes the fact that a certain (third party or extension) hardware is compatible with a machine. Usually this is defined by interfaces available on the basic unit, like serial, or parallel ports and especially any extension ports. With a different extension bus system as for example ISA, as defined by the original PC, there will be a hard time to use such ISA cards, thus hardware compatibility is not given.
Software compatibility defines if a certain program, written for a certain environment can run or not. For example an original IBM-PCXT without any video card will still boot and run quite well. That machine is by no doubt IBM compatible, still, running Windows 3.0 is somewhat fruitless - if it comes up at all. Too exotic? Well, plug in an MDA and you'll get the same result.
Maybe you remember the often lengthy list of sound cards, graphic cards, keyboards, mice and joysticks, a PC games did feature on the boxes during the 90s. All claimed to be PC games, but the user still had to verify a long list of features of versus his machine.
Software relies on what parts of the environment it accesses. Most obvious it's the instruction set written/compiled for. A program compiled for a 80186 program may not run on a 8088, but quite well on a 80286. Next it's how it interacts with other features this may be more hardware (like above games) or other software. Generally here three basic variations within the PC World:
a) Hardware compatible
Here a program accesses specific hardware on the lowest possible level. A program written to run direct with a CGA won't deliver any (useful) result without that card (or a close emulation).
b) BIOS compatible
BIOS compatible programs do all their machine specific bidding (beyond the CPU) via BIOS calls. Such programs may run on all machines that offer a BIOS with compatible calls to what the programmer expects, read an IBM-PC BIOS of a certain level. Since the PC-BIOS offers next to everything a character mode program needs, from keyboard and screen handling all the way to disk read and write, this is in some ways equal to what early home computers offered.
c) OS compatible
Here a program does all its machine handling (I/O) via a certain OS, like DOS or Windows. In reality, there's a split between these examples calling for two subclassifications
c.1) DOS compatible
A program running on DOS or similar operating systems can rely on a number of additional services, like the namesake disk management (working with files instead of devices)
c.2) WINDOWS compatible
These are programs that rely on Windows (*1) services. Unlike DOS, Windows includes not only more than rudimentary process support, but also independent interfaces for graphics, sound and a wide variety of input devices.
(*1 - This is as well true for other, alike environment, like GEM or GEOS)
In general, a program written for either of the above compatibility layers and following its mechanics will run on any machine offering the same kind of compatible interface. A DOS program written for an IBM-XT does as well run on a Sirus 1, a SIEMENS PC-D, a Tandy 1000 or a Nimbus PC-186. All these machines have been accused of being non-compatible at some point.
One major problem with real world applications is that the programmers did not really care much for clean interfaces - or couldn't because they needed certain capabilities. Like a game asking for a DOS environment and a EGA. Accessing that made it unable to run on a PC with MDA.
(Grandpa story: One nice example I 'converted' was Mult Edit 5.0 for the SIEMENS PC-D. Eventually the best ever editor of all times to the best ever PC. The PC_D was originally developed as Unix workstation, but made later on fully BIOS and DOS compatible - on a hardware somewhat like the NM-186. Multi Edit was well written, except for two minor quirks. It expected a memory mapped character display much like the MDA (or later) and it accessed the PC speaker directly. Everything else did run via BIOS or DOS calls. Thanks to the nice hardware debugging features of the PC-D it took me like half an hour to find and patch the two words used. Lucky me, the software was extreme clean and only had those two references.)
With restrictions like that, programs needed to be ported, or, if their architecture was advanced enough, a driver for different hardware on different machines compositions had to be added. Even for 'standard' machines separate drivers where needed for CGA, EGA and VGA (and more) where needed, as the publisher did want to cover more variations than only his 'standard'.
The major goal in the Windows (and GEM) development was to add abstraction layers for devices like human input (aka keyboards, mice, joysticks etc.), graphics and sound. So a all driver handling was taken away from the applications and bundled with the OS. Porting Windows (or GEM) thus didn't require porting the OS, but just adding new branches of machine/hardware detection and appropriate drivers. Application programs did not need to be changed in any way - as long as they are coded according to the OS guidelines. There where still programs that bypassed Windows or assumed specific settings without checking, but usually not from major developers.
A clean Windows 3.0 Program could run anywhere from an IBM-XT with 512 KiB and CGA to a PC-D with 960 KiB usable RAM, 720x350 B&W graphics and a 80186 or any Pentium 15 years later.
So called 'special' versions of Windows where not special at all, just the result of a Microsoft marketing. For one, MS offered the ability to customize loading screens so manufacturers could present 'their' Windows making the buyer believe he got something special. Second, MS offered deals to manufacturers. They got Windows at a greatly reduced price but it only included drivers for the hardware these companies offered. Special machines got even bigger discounts - like the NM-186.
The PC-186 was fully compatible with BIOS (AFAIK, there seam to be other information available), DOS and Windows. All clean applications written for either would run right away. Application accessing Hardware not available on the PC-186 would fail.