However, BIOS lived in a ROM.
Step 1: DOS-BIOS != ROM-BIOS
BIOS, as seen by DOS comes from DISK and is loaded with DOS. OEM users had to provide their BIOS as loadable code. A DOS capable machine thus only needed a boot loader, everything else was supposed to come from DISK during boot, like any OS does today.
On the IBM-PC on the other hand, IBM provided a ROM-BIOS. This was not specific designed for DOS, but more generic to cover whatever functions the mainboard offered, all DOS needed and many more (*1). As result, the BIOS code provided with PC-DOS - or MS-DOS for (mostly) compatible machines - was just a stub, directing DOS functions to ROM BIOS calls.
DOS never calls any ROM-BIOS function directly, it always goes thru the loaded BIOS, which provides a standard device driver (*2) for diskettes and hard disks using ROM-BIOS calls.
Step 2: Adding Option ROMs
Mainboard ROM could of course only support mainboard features (plus some very common/basic add-on cardsIBM assumed to be always present), thus its designers included an extension mechanism. After initializing all components it searches the address space between C0000h and EF800h (right below itself) in 2 KiB increments (C0800h, C1000h, ...) for a ROM signature of 55h, AAh. The next byte gives a length of that ROM. Following that is an entry point called by the motherboard BIOS so the ROM can initialize whatever necessary. That way a hard disk controller can install or hook the disk service at INT 13h (*3).
Step 3: Preboot Patch or Installation
When all Option-ROMs are scanned and activated, the BIOS calls INT 19h to boot. This loads the first block from whatever drive the installed INT 13h handler presents as boot drive. Usually this would be the MBR to load which in turn loads the OS.
Except, some disk software installs here a piece of software that patches the BIOS drivers parameter with values for an otherwise unknown drive, enabling the use of drives not known to the controller ROM. After patching the software continues with loading the real boot sector.
Some more sophisticated utilities install a RAM resident driver for disks that could not be adapted by simply setting the right parameters.
Step 4 : DOS Drivers
Of course, more hardware can be made present by loadign drivers with/after DOS - these can of course also replace existing BIOS code or install extensions.
For the question
Since the designers of the BIOS could not possibly have anticipated which exact hard drive a user would have installed, or which display, etc, how did BIOS know how to initialize these things?
BIOS only initialized whatever ROM drivers were provided. Either within the motherboard ROM, or due activation of all found Option-ROMs. A must for anything the BIOS/DOS needs to start, like keyboard, screen and disks, optional for everything else.
Additional hardware could of course be installed and initialized by loading a DOS device driver via CONFIG.SYS. Standard for many more sophisticated drives.
Was it possible to buy a hard drive (or something) that the BIOS wasn't written for?
Yes, quite often. Anything that didn't come with an interface card with a ROM driver that linked it in (or got camouflaged it as 'standard' drive) to be bootable (see above).
If booting wasn't a requirement, loading as DOS driver (which as well hooked BIOS) was the way taken. Quite common for SCSI drives, tapes, etc.
*1 - For example cassette handling.
*2 - Starting with DOS 2 - before that it was a set of hard coded calls.
*3 - Some also may hook INT 19h.