Welcome to the Retrocomputing Stack Exchange. First of all, if you have any reason at all to suspect the PSU, it's not advisable to connect it to the machine until further tests.
Buzzing could be just due to 50hz vibrations corresponding to the line frequency in the mounting or inside the transformer itself and is not necessarily critical, albeit annoying. You can maybe feel the vibrations if you put your hand on it. Maybe a mounting can be tightened, maybe there is nothing to do about it.
Buzzing can also be caused by arcing due to fx a loose connection on the primary side, which is worse (from a safety standpoint). The sound probably wont be as regular and you might be able to smell ozone in that case. Try to power it up outside its case (being careful not to touch anything on the primary side or to put it on something conductive) - it may help to locate the origin of the noise. Posting an image of the insides could assist people commenting on the issue as well.
I have not serviced this particular version of the Sinclair PSUs, but if they are like the ones I have they only contain the transformer, rectifier diodes and a smoothing capacitor. In that case an over voltage failure mode of the PSU is unlikely. If it contains active regulation, that's a failure point that could fail with over voltage - but as I recall, the regulation is ultimately done with a 7805 inside the Spectrum itself (that regulator can still fail with age, but that's another story as long as the focus is the PSU itself).
It is normal that a voltage measurement without the anticipated load is higher than the nominal voltage. 12-15V for a 9V supply sounds quite normal. But if you loaded it with say around 1A, 12V sounds a little on the high side - but the Spectrum's regulator should be able to deal with it. (Of course, the higher the voltage it has to dissipate the hotter it will get and the more strain is put on it and the shorter its lifespan may ultimately be.)
TL;DR - If there is no active regulation in the PSU, the voltage coming from it is likely fine. (That doesn't mean that it may not need service, fx degraded capacitors could lead to excessive ripple and noise/stability issues, but in general that it is not a condition that will damage the connected device.)