Allocate 64 KiB in Watcom C 16-bit DOS

Question

I'm trying to allocate a 64 KiB buffer in Watcom C 16-bit DOS. I'm using the "compact" memory model which defines the code segment to be limited to 64 KiB and addressed by near pointers (just offset), but the data segment to be addressed through far pointers (segment:offset).

From the documentation I also get the impression that using malloc in the compact memory model is simply an alias for _fmalloc.

However, when I try to allocate 64 KiB, or as close as I can get (1024 × 64 − 1, the maximum representable with a 16-bit size_t) the malloc fails. But when I allocate 64 KiB minus 32 bytes (i.e. 1024 × 64 − 32), the malloc succeeds.

How can I allocate 64 KiB (1024 × 64 − 1, i.e. 65535/FFFFh bytes)? And why does my malloc succeed when I subtract 32 bytes?

Answer 1

Typically memory allocators need some extra data in addition to the requested allocation size. This is for maintaining a list of allocated blocks on the heap, it's usually a linked-list node along with a couple of other bits of info like if the block is currently allocated or free. This tiny structure usually goes behind the pointer returned to you by malloc and is typically rounded up to some power-of-2 size to ensure pointers returned by malloc are aligned.

Because of this, you cannot allocate exactly the size of your heap because we need additional space for that block's link structure.

For example, eight malloc(8192) calls would also fail on the 8th call because each allocation would occupy, in your case, 8 KiB + 32 bytes.

Answer 2

I have not tried it but the following trick might work:

You can use halloc(0x1000, 0x10) to allocate a "huge" memory block of 0x1000 × 0x10 bytes (= 64 KiB) size. According to the manual this block will be 0x10-byte aligned if the second argument is 0x10.

halloc() can allocate more than 64K but returns a huge memory pointer, not a far memory pointer (see below).

In "Real Mode" ("normal" MS-DOS) the linear address is calculated as 0x10*segment+offset so if the offset is a multiple of 0x10 (if the address is aligned to 0x10 bytes) you can calculate a far pointer that points to the same memory using the following method:

You take the offset, divide it by 0x10 and add the result to the segment. Then you set the segment to 0.

Example:

union {
    struct {
        unsigned short off;
        unsigned short seg;
    } s;
    void far *ptr;
} u;

u.ptr = (void far *)huge_pointer; // Example: 0x1234:FED0
u.s.seg += u.s.off >> 4;
u.s.off = 0;
far_ptr = u.ptr; // should be 0x2221:0000

Both 0x1234:FED0 and 0x2221:0000 point to the same memory.

However there are two differences between the pointers:

• hfree() and similar memory management functions must be called with the "original" pointer returned by halloc(), not with the "calculated" one

• The difference between huge and far memory pointers is that far pointers do not allow crossing segment boundaries: A far pointer to X:Y can only access (0x10000 − Y) bytes of memory. So a far pointer to 0x1234:FED0 can only access the first 304 bytes of the memory block (0x10000 − 0xFED0 = 304) while a far pointer to 0x2221:0000 can access the first 64 KiB (note that halloc() can allocate more than 64K).

Answer 3

A community wiki answer because it includes a lot of useful evidence, but I seem to flub the numbers somewhere:

Per OpenWatcom, the definition of _fmalloc is this. Notably:

if( amt == 0 || amt > - ( sizeof( heapblk ) + TAG_SIZE * 2 ) ) {
        return( NULL );
}

Allowing for integer rounding and given that size_t is 16-bit, that asserts that if the block you asked for is larger than 65536 minus sizeof( heapblk ) + TAG_SIZE * 2 then the allocation will fail.

heapblk is a struct:

struct heapblk {
    tag                 len;                /* size of heap (0 = 64K) */
    heapptr             prev;               /* segment selector/offset for previous heap */
    heapptr             next;               /* segment selector/offset for next heap */
    freeptr             rover;              /* roving pointer into free list */
    unsigned int        b4rover;            /* largest block before rover */
    unsigned int        largest_blk;        /* largest block in the heap  */
    unsigned int        numalloc;           /* number of allocated blocks in heap */
    unsigned int        numfree;            /* number of free blocks in the heap */
    freelist            freehead;           /* listhead of free blocks in heap */
#if defined( __WARP__ )
    unsigned int        used_obj_any :1;    /* allocated with OBJ_ANY - block may be in high memory */
#endif
};

And from line 50 of the same file, TAG_SIZE is just sizeof(tag).

On line 106 tag turns out to be a typedef for unsigned int so it's also 16 bit.

heapptr is a union defined on line 115, of a __segment and a heapblk_nptr (a.k.a. a _WCNEAR). So it's two bytes as nptr means near pointer, and a __segment means 16 bit.

freeptr is a union defined on line 120, of a freelist_nptr and an unsigned int. So also two bytes.

freelist is a struct defined on line 125 consisting of a tag, and two freeptrs. So six bytes.

So a heapblk is 22 bytes of meaningful storage:

• 8 bytes for the four unsigned ints;
• 2 bytes for the tag (because it's another unsigned int);
• 6 more for the freelist;
• 2 for the freeptr; and
• 4 in total for the heapptrs.

That plus another 4 for the 2*TAG_SIZE is 26.

Therefore as posited elsewhere, you cannot allocate a full 64kb because some storage is reserved for heap management; however I seem to total that to 26 bytes, not 32.