The sound you hear is when DOS detects an error with reading/writing the disk and is trying to re-align the drive head by pushing it to the far ends. This is a pretty normal behavior, but usually indicates that your disk might be corrupted or the drive has a problem.
Seeing how you have a DOS loaded (whether DOS 3.3 or ProDOS) since you're able to type SAVE WHATEVER
and the disk drive spins, I'm assuming you booted from a good disk, or at least one that was good enough to load DOS. If you're trying to save to the same disk, it might be partially corrupted or no good. If you're trying to save to a different disk, then that disk might be bad and you should try a different one.
When talking about floppy disks being "no good", I'm referring to how the floppy disk you're using is most likely 30+ years old and the idea that they will degrade over time. Depending on how it was stored all these years, might be partially disintegrating, could have had mold or something else which causes it to not hold a good magnetic signal pattern anymore. As well, you should make sure that you're using one that is labeled as DS/DD (the 'DD' part referring to Double Density; a 1.2MB HD 5.25" IBM disk might not work very well).
To see if the disk is physically bad, you can inspect the disk by expanding your fingers like a 'V' in the center hole and carefully spin the disk while looking in the oval window. You should see a consistent brown/gray color and NOT see any spots or significant streaks. If you do see spots or discolorations, most likely the disk is no good anymore and will not work. If you don't see any particular problems, it might just be that the disk needs to be reformatted/erased.
If you can reformat the disk and the same problem happens again, then it might be your disk drive. It isn't too common for Apple //c drives to be out of alignment, but it is a bit more likely that the disk heads are dirty. If you're comfortable with opening up the disk drive again, the heads can be cleaned with Q-Tips and isopropyl alchohol.
That being said, a recap wouldn't hurt, but seeing how you got this far it is less likely to be the result of a bad cap.