I took a look at how fusetest does it, but first let me talk about the general theory. The key is to write a Z-80 subroutine that will run for any given number of T-states (Z-80 parlance for cycles). With that in hand a simple binary search can get an approximation of the number of T-states per frame. Wait for the vblank interrupt to trigger. Now set up the vblank interrupt handler to set a "hit" flag. Clear the hit flag. Call your the delay subroutine with some reasonable initial guess and then disable interrupts. If the "hit" flag is clear then it takes longer than your guess. Otherwise, your guess is longer than the frame. Adjust the guess appropriately and try again. By binary searching the guess you'll quickly get down to a value that is pretty close.
The routine in fusetest to burns cycles is called "delay". I've written a similar routine myself which you can read about here: http://48k.ca/beamhack3.html
There are simpler approaches if you have some idea how many T-states there can be a frame. For example, it might be that the vblank can only be every 1/60th of a second (for NTSC displays) or 1/50th a second for PAL. Getting to within 20 or 30 T-states of the actual number will be more than accurate enough to distinguish between the two which differ by thousands of T-states even at 1 Mhz.
Getting the exact count down to a single T-state is more difficult because of jitter. The fastest Z-80 instructions take 4 T-states. Luckily, HALT can be used to wait for an interrupt and it takes 4 T-states to run. HALT effectively puts the Z-80 into a tight loop where it will only continue to the next instruction when an interrupt occurs. But an interrupt can't be taken immediately -- it has to wait for the current instruction to finish. So by the time the Z-80 gets to run we can only know that we're within 0 to 3 T-states of the actually interrupt start.
The next interrupt will take place during our timing routine which makes matters worse. It could occur during any instruction in there which could be as many as 12 or 15 cycles depending on what the delay routine does. The inaccuracy at both ends means a simple search will only get to within 20 or so T-states of the exact time.
fusetest doesn't completely solve the problem, but it is clever about reducing the jitter on receiving the interrupt to 4 cycles. What it does first is get the frame time / 256 essentially by incrementing A register every 256 cycles. When the interrupt comes along A has the count.
Using that count it then delays for (A - 1) * 256 cycles and drops into a series of 63 INC A instructions. The interrupt will occur sometime during that "ramp" and since A is to to 0 it will give the (frame time % 256) / 4.
At this point it then assumes just assumes that the frame time is a multiple of 4 and reports the first count * 256 plus the second count * 4 as the number of T-states per frame.
Give or take. If you look at framelength.asm you'll see it actually reports the count - 0x8000. This is because there are more than 0x10000 T-states per frame which would inconveniently overflow our 16 bit BC register. And, now that I think about it, it also assumes there is at least 0x8000 T-states per frame. There's also a fair bit of fiddling to account for the time taken by the interrupt servicing and the instructions leading up to calling the delays and so on. Tricky and detailed, to be sure, but straightforward enough.
