Mixing small and big data models in 16-bit DOS & Watcom

Question

I want to write an application for DOS using small code/data model, but I would like to use far pointers for some selected memory blocks. I would like to use the OpenWatcom C++/16 compiler.

So, for most of the things I would like to have near pointers, but one array in the program I'd like to store using a far pointer, because it's big, accessed rarely, and I don't want to pollute my near heap with this large blob.

Normally I'm allocating the memory using malloc(), and this produces normal near pointers to my data from the "near" heap. For the larger array, I'd like to use _fmalloc() to allocate the data from the "far" heap.

Question number 1: Is it possible to mix malloc() and _fmalloc() in the same program using small memory data model?

I'm having doubts if that's possible, because the Watcom docs have this comment in clib.pdf:

The _nheapgrow function attempts to grow the near heap to the maximum size of 64K. You will want to do this in the small data models if you are using both malloc and _fmalloc or halloc. Once a call to _fmalloc or halloc has been made, you may not be able to allocate any memory with malloc unless space has been reserved for the near heap using either malloc, sbrk or _nheapgrow.

So I happily call _nheapgrow() on the beginning of my program:

int main() {
    _nheapgrow();
    uint8_t far* ptr = (uint8_t*) _fmalloc(20);
    printf("ptr: %08lX\n", ptr);
    ptr[0] = 0;  
    return 0;
}

The problem is that the call to _fmalloc() now returns a memory location which seems to overlap with other allocated memory blocks in the near heap (and the application data). In this example, it overwrites the "NULL pointer protection area", a 0x20-sized memory block filled with 0x01's which is supposed to guard against null pointer dereferences. This is reported as:

Question number 2: is this normal behavior, or is it a bug in the stdlib?

When switching to large data memory model, everything seems to be fine, since "near" heap isn't used at all (I think). Everything is aliased to _fmalloc(), _ffree(), etc, and the "near" variants are never called.

So in order to use _fmalloc(), am I forced to switch to large data memory model?

Answer 1

It was because of the wrong cast. The cast should be (uint8_t far*), not (uint8_t*).

Answer 2

While it would have been sensible and useful to say that a pointer-to-pointer cast which specifies neither a near nor far qualifier would be interpreted as yielding a far-qualified pointer if the original is far, and a near qualified pointer if the original is near, that's not what x86 compilers from that era generally do (there may have be exceptions). Instead, any pointer type which doesn't specify a qualifier will be interpreted as near when using the small or medium memory model, and as far when using the compact or large model. Conversion of a far pointer into a near pointer will throw out addressing information which would be vital except in cases where the address in question identifies a "near" object; converting the pointer back to a far pointer will result in the generating code replacing that information with default value that would be correct for "near" objects, but erroneous for anything else.

Incidentally, while many people seem to view the notion of "memory models" as an 8086 quirk, the concept could be usefully employed even on many systems today. For example, many small embedded programs on the ARM make heavy use of a few global objects, which would typically need to be accessed via pattern such as:

extern int x,y,z;
void test(void) { x = y + z; }

test:
  mar r0,__addresses12345
  ldm r0!,{r1,r2,r3}
  ldr r0,[r1]
  ldr r1,[r2]
  add r0,r0,r1
  str r0,[r3]
  bx  lr
  .align
__addresses12345:
  dw  y,z,x

The code above would take 28 bytes and 14 cycles, of which 18 bytes and 5 cycles would be used loading x, y, and z, while 10 bytes and 9 cycles would would perform actual work. If there were a "tiny systems" ABI which specified that R7 would be left pointing to the start of a dedicated globals area, and the variables were declared as being within that area, then code size would shrink from 28 bytes to 10, and execution time would shrink from 14 cycles to 9.

IMHO, it would have been useful for the C Standard to have defined "near" and "far" qualifiers, leaving details of how--if at all--they map to system concepts as Implementation Defined, but having the general semantics that there is a certain amount of "near" address space (possibly all of it) which can be targeted by either "near" or "far" pointers, and that "far" pointers may be able to access storage that isn't accessible by unqualified pointers (if any such storage exists). While the qualifiers may not have done anything useful on all systems, they could have been specified in a manner that would have made them harmless on systems where they weren't needed while making clear how they would need to be used to guarantee compatibility with systems that required them.