I'm one of the developers of the Rekursiv. Its biggest problem was its recursive nature. Basically if a page fault happened while executing a (microcode, recursive) instruction it wasn't possible to abort execution of the instruction, issue a memory fault and switch to a different thread while the fault was serviced. Instead, the entire processor halted while the host machine (Sun 3) serviced the page fault, and then the Rekursiv carried on from where it had left off.
Also, the fact that there were so many different memories made efficient use of RAM tricky, and there was no VMM on anything but the object-store (IIRC), so there was a distinct upper limit on the number of processes which could run.
If we'd been able to integrate the entire CPU on one chip I think it would have been faster than the Sun, but we couldn't and so inter-chip communication delays probably halved what could have been achieved for execution rate. That and the fact that the board had no access to I/O of its own made getting to and from disk pretty slow.
Finally, if any of us had had any actual previous experience designing high-speed digital circuits or even (gasp!) processors, we might have made it go a bit faster. But we didn't :-)