I know that BIOS is obsolete, hence why I'm asking here :-) BIOS needed to do things like start loading the OS, for which it (usually?) would have needed things like a hard disk driver.
However, BIOS lived in a ROM. Since the designers of the BIOS could not possibly have anticipated which exact hard drive a user would have installed, or which display, etc, how did BIOS know how to initialize these things?
Was it possible to buy a hard drive (or something) that the BIOS wasn't written for?
The IBM PC design makes a provision for option ROMs, which allow expansion cards to patch extensions into the BIOS.
That's how you get things like the IBM EGA card retrofitting new int 10h services that wouldn't be built into the BIOS until the IBM AT and the 1986 revision of the IBM XT BIOS, network cards adding netboot support, or hard drive controller cards adding support for things like booting off SCSI.
(And option ROMs are still a thing, even with UEFI. That's where some of the Thunderbolt/USB4 vulnerabilities came from. Firmware neglecting to whitelist which PCIe lanes will have their option ROMs initialized.)
However, that's also not the only workaround that was done in practice, because many machines had onboard IDE controllers with the BIOS only capable of addressing up to 504MiB (528 MB), and then 2GiB, and then 4GiB. (Though, technically, that last one is an OS problem that got patched in the BIOS instead.)
The solution without installing a new hard drive controller card was to use something like OnTrack Disk Manager (Wikipedia) or Micro House EZ-Drive which would install a custom boot sector that would patch the BIOS's int 13h routines before loading the operating system, since the BIOS only needs to be able to find the boot sector to load it, and the routines are patched by the time DOS needs to rely on them to access offsets beyond the limit.
For a time, it wasn't uncommon for hard drive manufacturers to include a copy of one of those pieces of software on floppy disk with their retail SKUs to ensure customers would be able to make use of the drive's full capacity. Wikimedia Commons has a photo of a Western Digital-branded EZ-Drive disk and the OnTrack page I linked includes a download for a Hitachi OEM release.
However, BIOS lived in a ROM.
No, but....
BIOS, as seen by DOS comes from DISK and is loaded with DOS. OEM users had to provide their BIOS as loadable code. A DOS capable machine thus only needed a boot loader, everything else was supposed to come from DISK during boot, like any OS does today.
On the IBM-PC on the other hand, IBM provided a ROM-BIOS. This was not specific designed for DOS, but more generic to cover whatever functions the mainboard offered, all DOS needed and many more (*1). As result, the BIOS code provided with PC-DOS - or MS-DOS for (mostly) compatible machines - was just a stub, directing DOS functions to ROM BIOS calls.
DOS never calls any ROM-BIOS function directly, it always goes thru the loaded BIOS, which provides a standard device driver (*2) for diskettes and hard disks using ROM-BIOS calls.
Mainboard ROM could of course only support mainboard features (plus some very common/basic add-on cardsIBM assumed to be always present), thus its designers included an extension mechanism. After initializing all components it searches the address space between C0000h and EF800h (right below itself) in 2 KiB increments (C0800h, C1000h, ...) for a ROM signature of 55h, AAh. The next byte gives a length of that ROM. Following that is an entry point called by the motherboard BIOS so the ROM can initialize whatever necessary. That way a hard disk controller can install or hook the disk service at INT 13h (*3).
When all Option-ROMs are scanned and activated, the BIOS calls INT 19h to boot. This loads the first block from whatever drive the installed INT 13h handler presents as boot drive. Usually this would be the MBR to load which in turn loads the OS.
Except, some disk software installs here a piece of software that patches the BIOS drivers parameter with values for an otherwise unknown drive, enabling the use of drives not known to the controller ROM. After patching the software continues with loading the real boot sector.
Some more sophisticated utilities install a RAM resident driver for disks that could not be adapted by simply setting the right parameters.
Of course, more hardware can be made present by loadign drivers with/after DOS - these can of course also replace existing BIOS code or install extensions.
Since the designers of the BIOS could not possibly have anticipated which exact hard drive a user would have installed, or which display, etc, how did BIOS know how to initialize these things?
BIOS only initialized whatever ROM drivers were provided. Either within the motherboard ROM, or due activation of all found Option-ROMs. A must for anything the BIOS/DOS needs to start, like keyboard, screen and disks, optional for everything else.
Additional hardware could of course be installed and initialized by loading a DOS device driver via CONFIG.SYS. Standard for many more sophisticated drives.
Was it possible to buy a hard drive (or something) that the BIOS wasn't written for?
Yes, quite often. Anything that didn't come with an interface card with a ROM driver that linked it in (or got camouflaged it as 'standard' drive) to be bootable (see above).
If booting wasn't a requirement, loading as DOS driver (which as well hooked BIOS) was the way taken. Quite common for SCSI drives, tapes, etc.
*1 - For example cassette handling.
*2 - Starting with DOS 2 - before that it was a set of hard coded calls.
*3 - Some also may hook INT 19h.
BIOS initially did not support hard drives at all.
Later on hard drive interfaces existed which had onboard ROM to replace the original implementation of BIOS disk interface to support that specific interface.
Since hard drive interfaces were at least de facto standards, so hard drives of different sizes and from different manufacturers were compatible with the interface, so the BIOS accessed them identically, just the actual cylinders, heads, and sectors per track varied depending on hard drive size.
These interfaces were later on integrated on motherboards so the motherboard BIOS supported the onboard hard drive interface directly.
First interfaces were XT class 8-bit interfaces with no integrated BIOS support. Then on AT computers there was MFM, ESDI, and AT Attachment bus (ATA) with integrated drive electronics (IDE) and these typically were directly supported by the BIOS, so in the 386 era the BIOS has the drive support while a simple ISA card with just a couple of logic ICs can implement an IDE interface for connecting the drives.
And yes, if you bought a SCSI card and SCSI hard drive, the BIOS does not know anything about it. It requires again support from the SCSI card to have onboard BIOS which installs itself on top of motherboard BIOS to implement support for accessing the SCSI hard drive over the SCSI interface.
Same thing with CD-ROM drives really. Even if there exists models that can be connected to ATA/IDE buses like hard drives, if the BIOS only supports hard drives, it does not understand that there are CD-ROM drives and thus booting from CD-ROM is then not possible. But again support for CD-ROMs and other ATAPI drives were added into BIOS so then it was possible to detect CD-ROM drives and ZIP/JAZ/LS120 drives on IDE bus and boot from them.
It was still possible to install a hard drive that is not supported - like for example on many occasions, a BIOS which is too old does not understand newer hard drives because they are larger. One classic limitation is the 504MB limit, since the BIOS could only initially access drives with up to 1024 cylinders, 16 heads and 63 sectors per cylinder. It was possible that it could detect the hard drive values and print them for the yser, but still if the cylinder count was larger than 1024 then it can't be used properly via the BIOS. And the hard disks can't provide an option ROM to fix this so software overlays were used to fix that. Sometimes a too large drive would simply hang the BIOS, so it was not possible to use it, and so hard drives had jumpers to limit their size for compatibility.
The BIOS also does not understand different display adapters. The BIOS had only support for MDA and CGA display adapters, so including EGA and later cards, they come with their own video BIOS to add support for them, and for the specific card installed.
Same with network adapters if they were required for booting, otherwise the drivers for the adapter would just make it work.
