2

Newer Windows and Dos binary executable files with the .exe extension have an MZ header in them and nowadays also a PE header.

But before these there used to also be .com binary executable files and they had no internal header, they were just raw x86 machine code.

But Intel's x86 processor family has been around for ages from the 8-bit era through to the 64-bit era with the instruction set and the memory model changing several times. Memory models had to do with how to access more memory beyond the processor's address bus could directly address. There used to be multiple pages of memory and segment registers

So back to .com files, did they use "real mode" or "protected mode"? Or were both used? If both were used was there any way to tell which was needed?

I want to disassemble some old .com executables and the tool asks me whether the files are real mode or protected mode.

(I did assembly and machine code programming on Z80 and Motorola 680x0 and switched to PCs in the 486 (32-bit) era but found the assembly syntax and segmented memory models ugly and didn't continue with assembly language or even understand the stuff about memory segments and such.)

  • 1
    Certainly it's possible for code written as real-mode code to execute in protected mode, in the V86 submode. – another-dave Apr 25 at 12:40
9

DOS programs always start in real mode (or an emulation thereof), so it’s best to start disassembling them assuming that. When disassembling, you should assume real mode, with 16-bit data and 16-bit addresses, until the code you’re disassembling changes that. The DOS-based disassemblers I’ve used generally know about the executable formats involved, and don’t ask. (They need to know about the formats to be able to set the segments up appropriately.)

In DOS, there are three executable layouts, and the extension doesn’t matter.

The first is the “.COM” layout, which is loaded into memory as-is after the PSP, with all free memory allocated to the process, and all segments set to point at the PSP. Execution starts at offset 0100h, which maps to offset 0 in the file.

The second is the device driver layout, which I won’t detail here.

The third is the MZ layout, with a header used to tell the loader how to load the executable. This header specifies how much of the executable to load initially, how much memory to allocate, the segment layout, relocations to perform, and where execution starts.

All three layouts leave plenty of room for a switch to protected mode, so ideally your disassembler should be able to recognise that.

Non-DOS executables have a real-mode DOS stub; if you disassemble assuming real-mode MZ, that‘s what you’ll see. Again, most disassemblers can interpret the format correctly and will open LE, NE, PE etc. appropriately.

The age of your executable can be a useful piece of information: protected mode DOS programs were unusual until at least 1993.

With the freeware version of IDA 7, you can analyse COM files by loading them as binary files, changing the loading offset to 0x100, and selecting 16-bit mode.

| improve this answer | |
  • I'm using Ghidra, which works with many executable formats and CPU architectures, but its loaders seem to ignore file extensions and detect file types only by their internal format. But as .com files have no internal format it doesn't recognize them, and I suppose these are not the kinds of binaries that are the primary target of Ghidra anyway. Dunno. My executable is an old ZX Spectrum emulator called Spec256 from the '00s or '90s. – hippietrail Apr 25 at 9:28
  • 1
    Right, Ghidra doesn’t know how to handle COM files unfortunately (although I imagine it would be possible to teach it how). Give the demo of IDA a try, it should work much better for this type of analysis. – Stephen Kitt Apr 25 at 10:19
  • I was just thinking, would these .com files then all use 16-bit instructions sets? I think DOS existed pre 16-bit and I think real .com files without MZ headers were still around in 32-bit DOS and Windows but maybe they ran in a special mode? Maybe this warrants a new question? (I'm teaching myself how to make Ghidra Loaders and .com seemed easy at first glance...) – hippietrail Apr 27 at 5:15
  • 1
    DOS didn’t exist pre-16-bit, it was always a 16-bit operating system. (See Can x86 processors run 8-bit applications?) It also didn’t make the migration to 32 bits. You should always assume 16-bit data and addresses, until the program being analysed issues instructions (or prefixes) to change that. One common feature was for 386+ programs to use the 32-bit registers in real mode, but you’d see the corresponding prefix. For Windows programs things are different, but they all have headers. – Stephen Kitt Apr 27 at 8:03
3

com files are not segmented (its just single segment). They have limitation that it can not cross 64K of code (filesize).

They always starts in real mode but I do not think there is any restriction to switch to protected from the code.

So while disassembling set real mode any switching from the code should be recognized by the tool. However that is improbable as in 64K of code there is not much room to make protected mode stuff as you need to make own OS like environment. There where used DOS extentions like DOS/4GW for this but newer saw it used along with *.com file.

Here an simple *.com source code in NASM that measures CPU clock (by RDTSC) I wrote ages ago:

[BITS 16]
[ORG 100h]

[SEGMENT .text]
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Main: ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
main:   push    word 0      ;fs ... segment 0
    pop     fs
    mov     ax,3
    int     16
    mov     dl,25
    mov     dh,dl
    call    kurxy

    mov     al,34h      ;set PIT freg. to 18,2 Hz
    out     43h,al
    sub     ax,ax
    out     40h,al
    out     40h,al

    lea     si,[txt0]
    mov     di,(22+12*80)*2
    call    print
    add di,40
    call    strnxt
    push    di
    add     di,24
    call    print
    pop     di

    call    sync        ;init counter
    rdtsc
    mov     [cntl],eax
    mov     [cnth],edx


l0: push    di
    call    sync
    rdtsc
    mov ebx,eax
    mov ecx,edx
    xchg    [cntl],eax
    xchg    [cnth],edx
    sub ebx,eax
    jnc r0
    dec ecx
r0  sub ecx,edx
    mov eax,ebx
    mov edx,ecx
    mov ebx,55
    div ebx

    lea si,[cache]
    call    num32
    call    print

    pop     di
    mov     ax,256
    int     16h
    jz  l0

    sub     ax,ax       ;key loop
    int     16h
    ret
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Time: ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
sync:   push    bx      ;Wait for next 55ms takt.,bl=current cnt
    mov     bl,[fs:46Ch]
syncl:  cmp     bl,[fs:46Ch]
    jz  syncl
    pop     bx
    ret
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;   TXT mode 3 library:   ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
kurxy:  pusha       ;dl = stlpec, dh = riadok   set kur.poz.
    mov     ah,2
    sub     bh,bh
    int 16
    popa
    ret
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Includes: ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
%include    'prn003.lib'
%include    'string.lib'
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Data: ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
txt0    db  'Pentium CPU CLK is:',0
    db  'KHz',0

cntl    dd  0
cnth    dd  0
cache:
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

and lib files:

prn003.lib:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;80*25 txt print ver: 1.0
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;print      ds:si => scr:di
;printl     text after call => scr:di , di = next line
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Text: ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
print:  pusha           ;ds:si >>> scr:di
    push    es
    push    word 0B800h
    pop     es
.l0:    lodsb
    or      al,al
    jz      .r0
    stosb
    inc     di
    jmp     .l0
.r0:    pop     es
    popa
    ret
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; End. ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
printl: mov [cs:.dat],si
    pop si
    push    ax
    push    es
    push    di
    push    word 0B800h
    pop     es
.l0:    mov al,[cs:si]
    inc si
    or      al,al
    jz      .r0
    stosb
    inc     di
    jmp     .l0
.r0:    pop di
    pop es
    pop ax
    add di,160
    push    si
    mov si,[cs:.dat]
    ret
.dat:   dw  0
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; End. ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

string.lib:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;String librrary ver: 1.0
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;strlea     lea (ds:si),[cs:txt after call]
;strlen     ax<=length [ds:si]
;strnxt     si<=next str [ds:si]
;strcmp     cmp strings [ds:si],[ds:di] ... set flags as cmp
;strmov     [ds:si] is copyied to [ds:di]
;strmovn    [ds:si] is copyied to [ds:di] di,si points to next string

;FPU:         fpu reg
;txt32      (2) [ds:si] txtdec => eax
;num64/32/16/8  (1) (+/-) edx+eax/eax/ax/al=>[ds:si] txtdec  meni asi 100b
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; Text: ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
strlea: pop si
    mov [cs:.dat],si
    push    cs
    pop ds
    call    strnxt
    push    si
    mov si,[cs:.dat]
    ret
.dat:   dw  0
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
strlen: push    cx      ;ax <<< length [ds:si]
    push    si
    sub     cx,cx
.l0:    lodsb
    inc     cx
    or      al,al
    jnz     .l0
    dec     cx
    mov     ax,cx
    pop si
    pop     cx
    ret
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
strnxt: push    ax      ;si = next str
.l0:    lodsb
    or      al,al
    jnz     .l0
    pop     ax
    ret
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
strcmp: pusha       ;cmp strings [ds:si],[ds:di]
.l0:    lodsb
    cmp al,[ds:di]
    jnz .r0 ;if <> then ret
    inc di
    or  al,al
    jnz .l0
.r0:    popa
    ret
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
strmov: pusha       ;mov  [ds:di] <<= [ds:si]
.l0:    lodsb
    mov [ds:di],al
    inc di
    or  al,al
    jnz .l0
    popa
    ret
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
strmovn:push    ax  ;mov  [ds:di] <<= [ds:si] and di a si points to
.l0:    lodsb       ;next string
    mov [ds:di],al
    inc di
    or  al,al
    jnz .l0
    pop ax
    ret
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
txt32:  push    si          ;[ds:si] txt => eax
    mov word [cs:.dat],10
    fild    word [cs:.dat]
    fldz
    sub ah,ah
    cmp [ds:si],byte '-'
    pushf
    jnz .l0
    inc si
.l0:    lodsb
    cmp al,'0'
    jb  .esc
    cmp al,'9'
    ja  .esc
    sub al,'0'
    mov [cs:.dat],ax
    fmul    st0,st1
    fiadd   word [cs:.dat]
    jmp .l0
.esc:   fistp   dword [cs:.dat]
    fcomp   st0
    mov eax,[cs:.dat]
    popf
    jnz .r0
    neg eax
.r0:    pop si
    ret
.dat:   dd  10
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
num64:  pusha           ;(+/-) edx+eax=>[ds:si txt];
    add si,20       ;limit is 999 999 999 999 999 999
    mov [ds:si],eax
    mov [ds:si+4],edx
    fild    qword[ds:si]
    mov dl,18
    jmp num8.num
num32:  pusha           ;(+/-) eax=>[si txt];
    add si,20
    mov [ds:si],eax
    fild    dword[ds:si]
    mov dl,10
    jmp num8.num
num16:  pusha           ;(+/-) ax=>[si txt];
    add si,20
    mov [ds:si],ax
    fild    word[ds:si]
    mov dl,5
    jmp num8.num
num8:   pusha           ;(+/-) al=>[si txt];
    add si,20
    cbw
    mov [ds:si],ax
    fild    word[ds:si]
    mov dl,3
.num:   mov di,si
    dec di
    fbstp   [ds:si]
    mov cx,10
    mov bx,'00'
.l0:    lodsb
    mov ah,al
    and ah,15
    shr al,4
    add ax,bx
    dec     di
    mov [ds:di],ax
    dec di
    loop    .l0
    inc di
    mov ah,' '
    cmp al,'8'
    jnz .r0
    mov ah,'-'
.r0:    mov [ds:di],ah
    sub dh,dh
    inc di
    mov si,di
    add si,19
    sub si,dx
    mov cx,dx
    push    es
    push    ds
    pop es
    rep movsb
    sub al,al
    stosb
    pop es
    popa
    ret
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.