# How can I detect signed integer overflow on the Game Boy?

Unlike a normal Z80, the Game Boy has no `CALL pe/po`, `JP pe/po` or `RET pe/po` instructions. I've been trying to figure out how to detect integer overflow on the Game Boy. (It doesn't have those instructions for the sign flag either but that can be done with `BIT 7`.)

This is what I've been trying to come up with but it's an absolute mess. After doing a few calculations on paper, such as `&81 - &02`, `&7F-&01`, etc. I came to the (possibly faulty) conclusion that overflow is equivalent to sign change xor Carry Flag. It almost works but there seems to be a problem. If A and B have opposite signs but they subtract to less than 7F the compare is wrong. The second picture shows an incorrect result.   Like I said, the source code is spaghetti, but I couldn't come up with a better way.

``````;compares A to B signed.
bit 7,a        ;returns nz if A is negative, z if positive.
call GetZeroFlag  ;stores this result
sub b
bit 7,a
push af
call TestForSignChange
jr z,NoSignChange
pop af
;at this point, if carry is clear, overflow occurred.
jr c,NoOverflow
jr OverflowOccurred
NoSignChange:
;at this point, if carry is set, overflow occurred.
pop af
jr nc,NoOverflow
OverflowOccurred:
;test bit 7 of A again. If A is positive, A < B. If A is negative, A >= B
bit 7,a
jr nz, GreaterThanSigned
LessThanSigned:
scf         ;set carry if less than, just like an unsigned compare does.
ret
GreaterThanSigned:
or a        ;clear carry if greater than or equal, just like an unsigned compare does.
ret
NoOverflow:
;Test bit 7 of A again. If A is positive, A >= B. If A is negative, A < B.
bit 7,a
jr nz, GreaterThanSigned
;I don't understand why this isn't "JR Z, GreaterThanSigned"
jr LessThanSigned

GetZeroFlag:
push af
push bc
push af
pop bc           ;get the flags into the C register
ld a,c
and %01000000    ;zero flag is where the 1 is.
ld (tempflags),a ;store the result in ram
pop bc
pop af
ret

TestForSignChange:
;compares bit 7 of A before and after the subtraction.
;If the value of bit 7 of A changed, return nz, otherwise return z.
push bc
push af
pop bc
ld a,c
and %01000000    ;ignore all but the zero flag
ld c,a           ;store the current state of the zero flag in C
ld a,(tempflags) ;get the state of the zero flag before the subtraction in A
cp c             ;compare it to the state of the zero flag after the subtraction.
pop bc
ret
``````
• Keep in mind, the Game Boy CPU is not a Z80, but like the Z80 an 8080 based design using Z80 style mnemonics. Oct 16 at 12:22
• I'm aware, I didn't know what to classify it as so I just call it a "Z80 derivative" even though it really isn't Oct 16 at 18:34
• hello, I would know, what assembler dialect this is written in. Can this be done with the nasm - the netwide assembler. I thinking nasm can make raw binary copies of program data...
– Jens
Oct 17 at 10:38
• @Raffzahn The question starts with ‘unlike a normal Z80’, so your comment is pretty redundant. Oct 17 at 11:02

You can determine overflow for addition and subtraction of two signed bytes A and B as follows. First flip bit 7 of both A and B. This will not affect the result of (8 bit) addition or subtraction, nor of the sign flag, but it will affect the carry.

Then for addition compute A+B. The overflow flag will be (carry == sign). For subtraction compute A-B. The overflow flag will be (carry != sign).

Example for subtraction:

``````; subtract signed byte in addrb from signed byte in addra
; result in B, overflow in carry flag.

; flip bit 7 of addrb and store in B
xor 80h
ld B, A

; addra in A and flip bit 7
xor 80h

; subtract B to compute (addra) - (addrb). move result to B
sub B
lb B, A

; test overflow: carry != sign.  carry flag holds result.
rla
rra
``````

A signed compare `(addra) < (addrb)` is much simpler:

``````; compare signed byte in addra with signed byte in addrb.
xor 80h
ld B, A
xor 80h
cp B

``````
• I've tried it but I can't get it to work. I really wanted it to work because your way is a LOT shorter, and it probably does work, but I'm most likely doing it wrong. Oct 17 at 0:27
• @puppydrum64 Added (untested) example code for subtraction. Addition would be similar.
– WimC
Oct 17 at 11:31
• Thanks, I'll test it this afternoon. I was using a custom macro to flip bit 7, which seemed to work correctly but may have altered the flags in a way I didn't intend. Oct 18 at 11:12
• It works! Tommy's is correct too, no disrespect intended to him. I prefer shorter code than longer code and this is much shorter and simpler. Which matters more on systems with limited space like the game boy Oct 18 at 21:31

Overflow occurs on addition when two numbers with the same sign add up to a number with a different sign.

It occurs on subtraction when two numbers with different signs produce a result with the same sign as the second number.

So the general rule is: a number begins on one side of zero; the add or subtract should move it further away from zero; somehow it ends up on the other side.

So for addition of `b` and `c`, I guess something like:

``````; Compute result, store in h.
ld a, b
ld h, a

; Test for the same sign in the two original numbers, store in l.
ld a, b
xor c
cpl
ld l, a

; Test for a sign difference with the result.
ld a, b
xor h

; Check the two sign differences.
and l
and \$80

jp z, no_overfliow
``````

That’s off the top of my head, untested and therefore possibly error-ridden, assuming all those instructions exist on the Game Boy’s moderately unique instruction set.

Alternatively, you could break it down into a series of sign tests; e.g. for subtraction the only possible causes of overflow are:

1. a positive number is subtracted from a negative number, and the result is positive; or
2. a negative number is subtracted from a positive number, and the result is negative.
• So far that seems to work! Your code there does the job. For subtraction all I had to do was remove the `CPL` after the xor c. I've essentially combined it with my other code and now I have a signed comparison routine. Now if only I could make it work for other registers besides B and C. I may just have to settle for only using those. Oct 16 at 18:35
• @puppydrum64 Note that you don’t need an (artificial) overflow flag for signed compare: if the operands have different sign, just swap them. Then do an unsigned compare. Another trick is to flip bit 7 of both operands A,B first and then subtract. Then the carry of A-B is set if and only if A < B for the original signed bytes.
– WimC
Oct 16 at 18:48