16

I am writing a tiny TSR program, and I want it to take as little memory while installed as possible. The memory footprint of every loaded DOS process, including a TSR, includes a data structure known as the Program Segment Prefix, taking 256 bytes. But since I realised that I don’t actually need all the information in the PSP while my TSR is running, I can put my program’s code in the PSP area to save some memory, effectively shrinking the PSP.

In particular, I don’t need the command line buffer; this allows me to slash the size of the PSP in half. I don’t use FCBs for I/O either (in fact, I don’t do any I/O at all), so I can cut the default FCBs too, further shrinking the PSP to 92 bytes. I think I might be able to reuse the JFT as well, by reducing the file handle limit to zero with an appropriate syscall (or just reducing the limit in the PSP directly), earning me 20 bytes back. (Maybe 19, if I want to guard against the odd overflow or off-by-one bug.) But then there are fields I am much less sure about. From offset 0x4e, there are some fields RBIL claims are ‘unused by DOS versions <= 6.00’, so maybe I could get away with overwriting them too, but then there are some rather cryptically described as being used by Windows 3.

My question is: how low can I go? How much of the PSP area can I get away with reusing for my own purposes with impunity? At which point should I expect the system to start acting up? And while I recognise that this makes the question rather fuzzy, I don’t necessarily mean the vanilla MS-DOS kernel here, but also clones, emulators (including Windows), programs like MEM, MSD, or anything else that might want to walk the process list and read the contents of PSPs. (In fact, I wouldn’t be too surprised if the answer were ‘zero, as long as you don’t invoke any syscalls’ as far as the vanilla kernel itself is concerned.) Also, since I am pretty sure I am not the first one to come up with this technique, I’ll also welcome pointers to other software using this trick.

1
  • 1
    It's been a while since I fiddled about with this, but I think my best results were gained by writing my TSR as a device driver and loading it from CONFIG.SYS.
    – Neil
    Commented Jun 1, 2021 at 20:10

2 Answers 2

12

Sidestepping the question: Allocate resident block without a PSP

My TSRs RxANSI and lClock do away with the PSP entirely.

First, the transient program frees the environment block. Then it relocates itself (process/PSP and all) to the top of the LMA, to avoid fragmentation later. This requires hacking the MCB of the target PSP so that it is self-owned, and correctly setting up the old process's parent return address, parent PSP, and the new parent's DOS call stack address (dword at offset 2Eh). Also take care to transfer the open file handles properly; you want the relocated process to retain its stdout and stderr handles. You also need to either close file handles only once, or DUP them with functions 26h, 55h, 45h, or 46h. Relocation is finished by terminating the old PSP with the setup that will return into the new PSP.

Next, the (relocated) transient part allocates a memory block for the resident part. (It can enable the UMB link and set up the allocation strategy so as to use UMBs, if found, so LH isn't needed.) Finally before terminating its process it hacks the resident block's MCB to be self owned. (The termination happens to close all the process's file handles too. If you do retain a PSP you generally want to close all file handles besides freeing the environment; imagine what happens if the user runs tsr > NUL - if the TSR does not free its handles then an SFT entry is leaked!)

As the resident block is self-owned after this (word [MCB segment:1] equal to MCB segment + 1), it shows up like a regular TSR in MEM output. In particular, its program name is read from the (zero-byte-padded) 8 bytes stored in the MCB at offset 8. However, this may also make software assume that there should be a PSP when there isn't. (A better check for PSPs is "self-owned memory block with the content starting with the two bytes CDh 20h" (an int 20h instruction, found at the start of every PSP).)

As indicated by the other answer, you can use a number of special values as the MCB owner. The value must be nonzero because zero indicates a free MCB. The value 8 is well-known for DOS data structures (first two name bytes contain "SD"), DOS code or excluded UMA ranges ("SC"), or other system usages. A solid check for system values can consider all values below 50h as indicating a "special" owner. Otherwise, and assumed by many programs for any non-zero non-8 value, the value is taken as a segment of a program. In particular, the program name is read from what is assumed to be an MCB at the segment one below the indicated owner value (as a segment).

Actual answer: Probing the "minimum PSP size" as indicated by the system

There is a place in DOS that gives an "authoritative" answer to the actual question: The function 31h handler enforces a minimum size for the PSP. This is 60h bytes in both MS-DOS 2 and in FreeDOS. Crucially, you can test this: create a dummy PSP and call function 31h with dx = 0. Like with PSP relocation, you can free the environment first. Like with an actual TSR with a PSP, you have to take care to close file handles if you DUP them. In my example I do not DUP the file handles but rather initialise the PHT to 20 minus-one entries in the test PSP.

I uploaded a test program to my repos as testpsps. To build it, check out the lmacros repo and this repo, then assemble with NASM. On a successful run it displays the amount of paragraphs that the test PSP retains after an interrupt 21h function 31h call with dx equal to zero. As expected this is 6 paragraphs running on the FreeDOS kernel 2042.

Ralf Brown's AMISLIB has an opinion on this, too

In the AMISLIB version 0.92 as released by Ralf Brown, one of the possible locations to which to install a newly resident program is the current PSP. It will preserve the first 40h (= 64) bytes of the PSP in that case, and copy the resident section to the PSP segment + 4. This is from AMIS.ASM, function $install_TSR:

 ;
 ; as a last resort, use our own PSP to store the code, and go resident
 ;
not_topmem:
        mov     ax,__psp
        mov     es,ax
        add     ax,4                    ; copy to offset 40h in PSP
        push    ax                      ; remember where we'll relocate
        add     resident_size,4
        mov     exit_func,31h           ; TSR rather than normal exit
        xor     ax,ax
        xchg    ax,es:[002Ch]           ; get and zero environment segment
        mov     es,ax
        mov     ah,49h                  ; since we will be going resident,
        int     21h                     ;   discard our environment
        pop     ax                      ; get back destination segment
 ;
 ; relocate TSR code into the PSP or UMB
 ; at this point, AX must be the segment at which to relocate
 ;
relocate_TSR_code:
        pop     cx                      ; get back TSR code size in paragraphs
        pop     bx                      ; get back TSR code segment
        push    ds
        mov     ds,bx
        ASSUME  DS:NOTHING
        mov     es,ax                   ; ES -> resident_seg
        ASSUME  ES:NOTHING
        xor     si,si
        xor     di,di
        mov     ax,16                   ; bytes per paragraph
        mul     cx                      ; get size in bytes
        or      dx,dx
        jnz     install_failed_pop      ; can only handle 64K at this time
        mov     cx,ax                   ; number of bytes to copy
        cld
        rep     movsb                   ; copy the TSR's code

And the example program NOTE.ASM uses a resident PSP that it creates in the first 40h bytes of its resident code segment. This "TSR PSP" has a particularly noteworthy comment:

;-----------------------------------------------------------------------
; Since we need a PSP, but might be loaded into a UMB or at the top of
; conventional memory, we make a copy of the all-important first 64 bytes
; of the PSP here.  After relocation, this copy will start at offset 0
;
TSR_PSP db 64 dup (?)

So Ralf Brown certainly was of the opinion that only the first 64 bytes of the PSP really mattered.

2
  • 2
    "In particular, its program name is read from the (zero-byte-padded) 8 bytes stored in the MCB at offset 8" Err, yes, but. This is only true for DOS>4. Any DOS prior to 4 left that area simply untouched. So anyone about to fetch a program name from MCB:8 must check for DOS>4 first.
    – Raffzahn
    Commented May 31, 2021 at 18:33
  • 2
    Or risk displaying garbage...
    – user21618
    Commented Apr 14, 2022 at 20:36
5

And while I recognise that this makes the question rather fuzzy, I don’t necessarily mean the vanilla MS-DOS kernel here, but also clones, emulators (including Windows), programs like MEM, MSD, or anything else that might want to walk the process list and read the contents of PSPs.

It doesn't make it fuzzy, but rather way to broad. Beside, one can never be sure about everything.

In fact, I wouldn’t be too surprised if the answer were ‘zero, as long as you don’t invoke any syscalls’ as far as the vanilla kernel itself is concerned.

Jup, to my knowledge that's the case. At least for plain (single tasking) DOS.

Also, don't forget to release the environment (*1).

If your goal is absolute minimization, then it would be best to request (see below) a chunk of RAM right the size of your resident part, move it over to that area and simply terminate your original program, freeing up the absolute maximum of RAM. Except, the newly searched block is as well marked as belonging to this, so it will get freed as well. To avoid this we have to 'gift' it to some other program.

Ownership is marked withing the Memory Control Block (MCB), a 16 byte structure right before the memory block in question. At offset 01h the memory handle (segment) of the PSP of the owning program is recorded. So changing that transfers ownership (bad hack, but there's no other way). Changing it to 0000h will mark it as free (*2).

There are usually three strategies:

  • Make it 'self' owned
  • Make it 'stranger' owned
  • Gift it to DOS

Self Owned means that ownership is given to a never existing program at this very location, so simply storing its own segment handle into that field.

Stranger Owned is done by giving it a handle that otherwise hardly exists, like 0FFFFh, or 00004h (Which is a nice way to call/mark it as a BIOS extension)

DOS Owned is done by giving it a value of 0008h, as that is used by DOS for all memory it allocates.

Self Owned is a somewhat common way to handle it. Personally I prefer DOS owned or BIOS owned.


Memory Allocation Strategy.

Getting the right Block is a bit of a problem. One would like to get the smallest fitting block but not want to take it away from the largest chunk. Luckily DOS offers exactly a function for that: Setting allocation strategy. It allows to define how memory blocks are to be searched for.

Parameters available are

  • Basic Strategy
    • (BX=00h) First Fit - First Block to fit
    • (BX=01h) Best Fit - Smallest block large enough to fit
    • (BX=02h) Last Fit - Highest memory address block to fit
  • Location
    • (BX=00h) Search Low Mem only (below 0A000h)
    • (BX=40h) Search High Mem only (above 0A000h)
    • (BX=80h) ​Search both with High first

One may think that best fit high first would make a good choice to deliver, but that may give a block in low memory, when there is still high memory available (*3), so it should be done in a two step process:

    MOV   AX,05800h ; Function 58h, subfunction 0h
    INT   21h
    PUSH  BX
    MOV   BX,041h   ; Best Fit High Only
    MOV   AX,05801h  ; Set it
    INT   21h
    MOV   BX,num_of_paras_needed
    MOV   AH,048h
    INT   21h       ; Get the mem
    JNC   DONE
    MOV   BX,001h   ; Best Fit Low Only
    MOV   AX,05801h  ; Set it
    INT   21h
    MOV   BX,num_of_paras_needed
    MOV   AH,048h
    INT   21h       ; Get the mem

DONE:
    POP   BX        ; Get original strategy
    PUSHF           ; Save flags if we made it or not
    PUSH  AX        ; Save the segment if we got one
    MOV   AX,05801h  ; Set it again
    INT   21h

    POP   AX
    POPF
    JNC   GOT_MEM   ; Got it?

; Oops, no memory found
; Error in AX
    Do whatever error processing is needed - or simply terminate.

GOT_MEM:
; Now move the TSR and so on.

*1 - Something every program should do after reading whatever it needs

*2 - And using a random Number may result in randomly being assigned.

*3 - Keep in mind that implementations may differ.

6
  • 1
    "One may think that best fit high first would make a good choice to deliver, but that may give a block in low memory, when there is still high memory available" -- Actually with a "high then low" strategy (bitmask 80h set), DOS first searches all UMBs and assigns one if found. Only if none is found in the UMA then the LMA is searched as another area. Anyway, you're failing to preserve and then enable the UMB link (function 5802h/5803h). And unlike my TSR examples as referenced in my comments, you're not recommending to relocate the original process so your allocation may fragment memory.
    – ecm
    Commented May 31, 2021 at 14:54
  • @ecm Yes and no. The behaviour to finish after UMA search is true, but implementation specific - question was to do something that will work in any possible system (I know, way wo broad, but still). And I'm not sure what "you're not recommending to relocate the original process" is supposed to say and how that should fragment memory.
    – Raffzahn
    Commented May 31, 2021 at 18:30
  • 1
    @Raffzahn: An example: Say you have no (large enough) UMBs free. Your environment is loaded to 1000h:0 and spans about 1 KiB. So your process is loaded to 1040h:0 and its transient setup takes up, say, 8 KiB. You want to install a resident block that needs 2 KiB. The only (large enough) free space is that directly behind your process, at address 1240h:0. You can free your environment but it's too small for your resident to fit in the resulting gap. If you now use First or Best Fit to allocate the resident block it goes to 1240h:0. Your transient terminates, leaving a gap of 9 KiB at 1000h:0.
    – ecm
    Commented May 31, 2021 at 19:07
  • 2
    @Raffzahn: How would you avoid leaving such a gap after installation of your resident? I don't know how to make this clearer. If you have used memory allocated (to DOS, shell, etc) up to segment 1000h, then a 9 KiB free gap, and then a 2 KiB resident program, and then some 500ish KiB free after, your free memory space is fragmented. That's bad news if you want to maximise the size of the largest free block.
    – ecm
    Commented May 31, 2021 at 19:43
  • 1
    @Raffzahn I prefer first to try to allocate with Int 21/AH=58h,BX=41, and only when it fails, use conventional LMB TSR installation, see AllocUMB.
    – vitsoft
    Commented Jun 1, 2021 at 18:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .