TL;DR: 186 machines are different
DOS is not restricted to 640 KiB, but may use the full 1 MiB of real mode address space as RAM.
Also, 80186 based PCs (*1)have usually only limited IBM-PC compatibility (*2). While all plain DOS software will run fine, any direct hardware access may or may not work. Most prominent is being direct video writes (*3). Additionally many software checking to distinguish 8088 based (XT) systems from 80286 based (AT) machines mistake the 80186 often as 286 and conclude running on an AT type hardware, expecting feature (like A20/HMA) which are not present on a 186.
Last but not least, writing to (imagined) HMA on any real mode x86 (*4) will most likely crash the system in a reliable manner.
The Questions
Can I have more than 640k conventional RAM under some DOS?
Not just 'some' DOS, but every DOS. DOS can use all physical memory there is on a real mode x86. It's the hardware that defines how much of the possible 1024 KiB are available as conventional memory. See as well Stephen's fine answer about the topic.
I'm looking at this 80186 PC that has RAM at address A0000. Video RAM starts at B0000 as expected for a Hercules; A0000 is just more main memory.
It was quite common for (early) non-IBM x86 PCs to offer more RAM. For example the Tandy 2000 could give up to 768 KiB of DOS-RAM (*5) while the SIEMENS PC-D offered up to 960 or 992 KiB, depending on BIOS version (*6). Both being 80186, but the same is true for 8086 machines. A good example being the Seattle Computer Product's S100 based 8086 CPU board - the very computer MS-DOS was developed on.
There's no bios extended data near 9F000 either; the bios extended data area is located somewhere above F4000.
Which is perfectly normal, as EBDA can resist anywhere in memory. 9F000h is only the default location on IBM PS/2 machines. The segment of EBDA is held in a word BDA at 40:0E
Can I just change the early boot sequence to patch the bios data area to say memsize = 704 instead of 640 before DOS starts, or is that nonsensical despite the memory literally being there.
DOS will use whatever amount the provider specific part of BIOS reports as paragraphs(*7) If no amount provided, MS-DOS will scan memory starting at 32 KiB (08000h) in paragraph sized steps. See for reference DOS 2's SYSINIT.ASM, documented in SYSINIT.TXT.
- Systems with writable memory mapped I/O, like video cards, will always provide that value.
- PC-DOS uses the BDA value also reported by INT 12h
Actually trying it on FreeDOS [...]
Well, the FreeDOS kernel is, or better can be compiled for 80186 use and should run file even on less than 100% IBM compatible machines. The same is not true for drivers and applications. You need to only load drivers that work with a real mode PC, usually those that work fine on an 8088.
What's with the XMS memory? That's just high memory (100000-10FFEF).
Not really, as the 80186 can not access memory above 1 MiB. Thus there can be no XMS. That's a feature only 286 and later have. This looks rather like a DOS configured to assume this being a 286 (*8).
I'm thinking there's some slight problem with this. The system freaks out when I load the high memory driver.
Well, most likely you're using drivers not made for that computer and/or CPU, drivers expecting a direct IBM-PC clone and a 286 or later, not recognizing being run on a 186.
An 80186 does not and can not have a HMA.
Accessing HMA works only on a 286 and later, due them providing more address lines than a 8086 (*9). The 80186 does not have those additional address lines. Using an address of 0FFFFh:10h, which on a 286 would result in absolute address 100000, the first usable HMA byte, will on a 186 warp and produce address 00000h.
As a result any write to HMA will either destroy the interrupt vectors, BDA, DOS data or DOS itself. Installing an HMA driver will of course put up some data structures, right there, reliably crashing the system with the next occurring interrupt.
*1 - Before the 286 was settled as standard for better-than-XT PCs, several companies used the 80186 with it's improved performance and high integration, much like Intel intended. Wikipedia has a shortlist of the most common 80186 based PC.
*2 - Plain 186 PC need in addition to be distinguished from such featuring IBM-PC like external components (VGA, interfaces) as well as similar SoC, like the NEC V40, based on a core with an 186 compatible instruction set, but IBM-PC like peripherals - they may add another level of confusion when identified as 186 instead of their exact variant
*3 - A bad habit forced due DOS prior to DOS2.0 and ANSI.SYS not really offering a device independent video interface, so BIOS access had to be used. Except, the original PC being already a rather slow machine and video output using BIOS was not exactly great.
*4 - That is any 8086, 8088, 80186, 80188, V20, V30 or similar
*5 - The Tandy 2000 is often seen providing 704 KiB using a driver that reserves the area at 0B0000h for an MDA emulation to enable higher IBM-PC compatbility.
*6 - The PC-D could have physical 1024 KiB of RAM, but top 64 were reserved by BIOS for its own code as well as an optional (never produced) colour graphics card. Later BIOS versions freed those 32 KiB,
*7 - This may differ from the amount provided by ROM-BIOS via INT 12h.
*8 - Some simple CPU test routines identify 186 as 286. Mostly done by programmers not aware of other than close IBM clones.
*9 - And the fine A20 gate disabled. See the OSDev-Wiki for programming details.
mem /d
might reveal more about how DOS allocates memory.