Why does the .z80 emulator-snapshot format have separate fields for the 8-bit refresh register and bit 7 of R?

Question

I've been looking at the .z80 file format for Sinclair ZX Spectrum games.

Most docs discussing the format include pretty much this wording:

    Offset  Length  Description
    ---------------------------
    11      1       Refresh register (Bit 7 is not significant!)
    12      1       Bit 0  : Bit 7 of the R-register

I'm assuming "Refresh register" and "R-register" refer to the same thing.

I know that only the low 7 bits of the R register increment.

But is the 8th bit actually used? Is its behaviour undocumented or can it only be altered by loading it with a value from the accumulator that has it set, which is used for some kind of software tricks?

I assume a sequence of events like this:

1. Originally people thought R bit 7 was never used and always 0 and the format documented that offset 11 bit 7 should be ignored.
2. Some emulators put junk at offset 11 bit 7.
3. It was discovered that some software did use R bit 7 after all.
4. A decision was made that just changing the documentation to make offset 11 bit 7 be meaningful would cause problems.
5. The format was extended to include a new field that specifically held the contents of R bit 7 at offset 12 bit 0.

But I can't find anything saying there was ever a .z80 spec without this meaning for offset 12 bit 0.

Is there an even more convoluted reason? Some kind of compatibility with older snapshot file formats?

Do some emulators actually use offset 11 bit 7? I have at least one Manic Miner .z80 that has offset 11 bit 7 set but offset 12 bit 0 unset.

Answer 1

The .z80 format comes from the Z80 emulator by Gerton Lunter. He released some documentation about the file formats used in it, and regarding offsets 11 and 12, this is what the manual says:

.Z80 FILES:
-----------

The old .Z80 snapshot format (for version 1.45 and below) looks like
this:

    Offset  Length  Description

    0       1       A register
    1       1       F register
    2       2       BC register pair (LSB, i.e.  C, first)
    4       2       HL register pair
    6       2       Program counter
    8       2       Stack pointer
    10      1       Interrupt register
    11      1       Refresh register (Bit 7 is not significant!)
    12      1       Bit 0  : Bit 7 of the R-register
                    Bit 1-3: Border colour
                    Bit 4  : 1=Basic SamRom switched in
                    Bit 5  : 1=Block of data is compressed
                    Bit 6-7: No meaning

From this, it can be assumed that the distinction from bit 7 and the other bits was there since the first public version of the format. OTOH, the Z80 emulator offers the option to not emulate register R, so it could be that the first internal versions of the .z80 format didn't even include a place to store register R. This may be deduced by the following paragraph, in which he mentions that offset 12 could have a FF value in it.

Because of compatibility, if byte 12 is 255, it has to be regarded as
being 1.  After this header block of 30 bytes the 48K bytes of Spectrum
memory follows in a compressed format (if bit 5 of byte 12 is one)....

About which software uses bit 7 of R: it is mainly used as a software pending interrupt flag. Instead of using the interrupt handler to do tasks, the handler is a minimal one which only sets bit 7 of R:

push af  ;or ex af,af' if alternate AF is not being used
ld a,128 ;or any other value >= 128
ld r,a
pop af   ;or ex af,af'
ret

(Note that this handler doesn't enable interrupts upon exit as it would normally be required. More on this, later). This handler is fast enough to not disturb most time sensitive routines, such as loaders or beep engines, which can run with interrupts enabled. Later on, when there is time ahead, the program just loads A with current contents from R (ld a,r). This operation also sets some bits in the flag register. More precisely, bit 7 of R is copied to the sign flag, so checking if an interrupt has been triggered in the past can be performed by executing:

ld a,r
jp m,InterruptIsPending

The InterruptIsPending routine should clear bit 7 of R and perform whatever tasks are relevant to do every 20ms (perhaps a play tick routine to produce AY-3-8912 music) and enable interrupts again.

I know that, for instance, the ZX Spectrum port of King's Valley uses this technique to know when an interrupt has been triggered so it calls a dirty rectangle routine for sprite fast updating. I myself have used this technique to implement a custom loading routine that allows loading interlaced pictures (those using both normal and shadow screens and are swapped at every vertical retrace to make them look as a hicolour resolution picture) and show them in interlaced form as they load (so, the loader has to swap screens at every vertical retrace interrupt). This is the relevant portion:

                ;IM 2 just toggles a flag at bit 7 of R to notify a vretrace has just happened
NewIM2          ex af,af'  ;4
                ld a,255   ;7
                ld r,a     ;9
                ex af,af'  ;4
                ret        ;10
                           ;Total: 34 cycles

Then, at several "safe" places in the load routine, I perform the R test. For example, just after a byte has been loaded:

;; LD-NEXT
L05C2:  ld a,r                  ;after storing a byte, check if a vretrace has happened.
        call m,SwitchScreens

The called routine performs the required operation AND enables interrupts again (this way, interrupts are not enabled until the delayed service routine has been executed)

                ;Alternate registers have already been set up for this.
                ;See beginning of ld_lacescreen
SwitchScreens   exx
                ld a,e
                xor d
                ld d,a
                out (c),a
                exx
                ld r,a
                ei
                ret

Answer 2

About the R register on a real Z80:

But is the 8th bit actually used?

Yes, it's freely available and won't be touched by any instruction except loading R.

Is its behaviour undocumented or can it only be altered by loading it with a value from the accumulator

It's well documented and can be used as assumed. When loaded all 8 bits from A are stored in R - and likewise returned when A is loaded from R. Thus it can be used for example as 'secret' flag to store a single bit value.

which is used for some kind of software tricks?

Software (like above) as well as hardware. After all, it gets put on the bus in each M1 cycle during refresh, thus it may also be used as a singe-bit output port.

---

About emulated use:

Is there an even more convoluted reason? Some kind of compatibility with older snapshot file formats?

I do not have any hard evidence, only an educated guess based on why an emulator might want to handle them differently.

In Ye Olde Days (tm) computers weren't many thousand times faster than the emulated target, so it was a good idea to keep the instruction path needed to perform short - especially at the core points like instruction decoding.

When trying to emulate a Z80, the R register needs to be incremented with every M1 cycle. That's essentially every instruction as well as any prefix within. To handle R like on a real Z80, the op code fetch would have to

• select bit 7,
• save it,
• increment R,
• mask out bit 7 and
• reinsert the saved value.

Depending on the CPU the emulator was made for, this can be many operations.

Now, if Bit 7 is handled separately, R can be defined as a simple 8-bit unsigned integer and operation during M1 is reduced to

• Increment R (value at offset 11)

That's about as short as it can get. All other handling is postponed to the times R get transferred from or to A. When setting, Bit 7 will be selected and stored at offset 12 (*1), while reading means combining these two values into one and storing in A.

Even in tight programs these instructions occur rather seldom, so this optimization will speed up emulation notably.

File formats are often direct images of memory structures, thus the 'convoluted storage' may simply be an artefact of how the emulator was optimized.

Do some emulators actually use offset 11 bit 7?

Not really. It simply happens. As the documentation says: don't care. It's an artefact. Nice but meaningless.

I have at least one Manic Miner .z80 that has offset 11 bit 7 set but offset 12 bit 0 unset.

Well, this seams to be simply a side-effect of the mentioned optimization. When that snapshot was taken, R had already (at least once) overflown into bit 7.

---

*1 - Using bit 0 to store bit 7 seems somewhat odd at first, but it allows extraction and insertion with shift instruction.

To extract, the (source) value of A is first stored at location 11, then shifted right into carry (or another register) and stored at location 12.

For example on a 8086 this can be done in 3 machine instructions

                  * AL containing the value of Z80 register A
    SHL AX        * Extract the value of bit 7 to AH
    SHR AL        * Reconstruct value for storing
    MOV AX,loc11  * storing both in Loc 11/12

(Shifting back and using a 16 bit store is faster than two 8 bit store instructions - even with misalignment.)

To restore it, the value from location 11 is shifted right again, then carry (or an upper byte, depending on the CPU used) is loaded and inserted per shift left.

For example on a 8086 this can be done in 3 machine instructions:

    MOV AX,loc11  * Load location 11 in AL and 12 in AH
    SHL AL        * move one bit up to prepare for insertion
    SHR AX        * Insert bit 7 into AL, now containing the new value of A