Unlike a normal Z80, the Game Boy has no CALL pe/po, JP pe/po or RET pe/po instructions. I've been trying to figure out how to detect integer overflow on the Game Boy. (It doesn't have those instructions for the sign flag either but that can be done with BIT 7.)

This is what I've been trying to come up with but it's an absolute mess. After doing a few calculations on paper, such as &81 - &02, &7F-&01, etc. I came to the (possibly faulty) conclusion that overflow is equivalent to sign change xor Carry Flag. It almost works but there seems to be a problem. If A and B have opposite signs but they subtract to less than 7F the compare is wrong. The second picture shows an incorrect result.

a positive, b positive

overflow failure

a positive, b negative

Like I said, the source code is spaghetti, but I couldn't come up with a better way.

;compares A to B signed.
    bit 7,a        ;returns nz if A is negative, z if positive.
    call GetZeroFlag  ;stores this result 
    sub b
    bit 7,a
    push af
        call TestForSignChange
        jr z,NoSignChange
    pop af
    ;at this point, if carry is clear, overflow occurred.
    jr c,NoOverflow
    jr OverflowOccurred
    ;at this point, if carry is set, overflow occurred.
    pop af
    jr nc,NoOverflow
    ;test bit 7 of A again. If A is positive, A < B. If A is negative, A >= B
    bit 7,a
    jr nz, GreaterThanSigned
    scf         ;set carry if less than, just like an unsigned compare does.
    or a        ;clear carry if greater than or equal, just like an unsigned compare does.
    ;Test bit 7 of A again. If A is positive, A >= B. If A is negative, A < B.
    bit 7,a
    jr nz, GreaterThanSigned
    ;I don't understand why this isn't "JR Z, GreaterThanSigned"
    jr LessThanSigned
    push af
        push bc
            push af
            pop bc           ;get the flags into the C register
            ld a,c
            and %01000000    ;zero flag is where the 1 is.
            ld (tempflags),a ;store the result in ram
        pop bc
    pop af
;compares bit 7 of A before and after the subtraction.
;If the value of bit 7 of A changed, return nz, otherwise return z.
    push bc
        push af
        pop bc
        ld a,c
        and %01000000    ;ignore all but the zero flag
        ld c,a           ;store the current state of the zero flag in C
        ld a,(tempflags) ;get the state of the zero flag before the subtraction in A
        cp c             ;compare it to the state of the zero flag after the subtraction.
    pop bc
  • Keep in mind, the Game Boy CPU is not a Z80, but like the Z80 an 8080 based design using Z80 style mnemonics.
    – Raffzahn
    Oct 16, 2021 at 12:22
  • I'm aware, I didn't know what to classify it as so I just call it a "Z80 derivative" even though it really isn't Oct 16, 2021 at 18:34
  • 1
  • hello, I would know, what assembler dialect this is written in. Can this be done with the nasm - the netwide assembler. I thinking nasm can make raw binary copies of program data...
    – Jens
    Oct 17, 2021 at 10:38
  • @Raffzahn The question starts with ‘unlike a normal Z80’, so your comment is pretty redundant. Oct 17, 2021 at 11:02

2 Answers 2


You can determine overflow for addition and subtraction of two signed bytes A and B as follows. First flip bit 7 of both A and B. This will not affect the result of (8 bit) addition or subtraction, nor of the sign flag, but it will affect the carry.

Then for addition compute A+B. The overflow flag will be (carry == sign). For subtraction compute A-B. The overflow flag will be (carry != sign).

Example for subtraction:

; subtract signed byte in addrb from signed byte in addra
; result in B, overflow in carry flag.

; flip bit 7 of addrb and store in B
ld A, (addrb)
xor 80h
ld B, A

; addra in A and flip bit 7
ld A, (addra)
xor 80h

; subtract B to compute (addra) - (addrb). move result to B
sub B
lb B, A

; test overflow: carry != sign.  carry flag holds result.
adc A, 0

A signed compare (addra) < (addrb) is much simpler:

; compare signed byte in addra with signed byte in addrb.
; carry set if and only if (addra) < (addrb).
ld A, (addrb)
xor 80h
ld B, A
ld A, (addra)
xor 80h
cp B

  • I've tried it but I can't get it to work. I really wanted it to work because your way is a LOT shorter, and it probably does work, but I'm most likely doing it wrong. Oct 17, 2021 at 0:27
  • @puppydrum64 Added (untested) example code for subtraction. Addition would be similar.
    – WimC
    Oct 17, 2021 at 11:31
  • Thanks, I'll test it this afternoon. I was using a custom macro to flip bit 7, which seemed to work correctly but may have altered the flags in a way I didn't intend. Oct 18, 2021 at 11:12
  • It works! Tommy's is correct too, no disrespect intended to him. I prefer shorter code than longer code and this is much shorter and simpler. Which matters more on systems with limited space like the game boy Oct 18, 2021 at 21:31

Overflow occurs on addition when two numbers with the same sign add up to a number with a different sign.

It occurs on subtraction when two numbers with different signs produce a result with the same sign as the second number.

So the general rule is: a number begins on one side of zero; the add or subtract should move it further away from zero; somehow it ends up on the other side.

So for addition of b and c, I guess something like:

; Compute result, store in h.
ld a, b
add c
ld h, a

; Test for the same sign in the two original numbers, store in l.
ld a, b
xor c
ld l, a

; Test for a sign difference with the result.
ld a, b
xor h

; Check the two sign differences.
and l
and $80

jp z, no_overfliow

That’s off the top of my head, untested and therefore possibly error-ridden, assuming all those instructions exist on the Game Boy’s moderately unique instruction set.

Alternatively, you could break it down into a series of sign tests; e.g. for subtraction the only possible causes of overflow are:

  1. a positive number is subtracted from a negative number, and the result is positive; or
  2. a negative number is subtracted from a positive number, and the result is negative.
  • So far that seems to work! Your code there does the job. For subtraction all I had to do was remove the CPL after the xor c. I've essentially combined it with my other code and now I have a signed comparison routine. Now if only I could make it work for other registers besides B and C. I may just have to settle for only using those. Oct 16, 2021 at 18:35
  • @puppydrum64 Note that you don’t need an (artificial) overflow flag for signed compare: if the operands have different sign, just swap them. Then do an unsigned compare. Another trick is to flip bit 7 of both operands A,B first and then subtract. Then the carry of A-B is set if and only if A < B for the original signed bytes.
    – WimC
    Oct 16, 2021 at 18:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .